Multiple ultrasound image registration system, method and transducer

ABSTRACT

An ultrasonic imaging system includes an ultrasonic transducer having an image data array and a tracking array at each end of the image data array. The tracking arrays are oriented transversely to the image data array. Images from the image data array are used to reconstruct a three-dimensional representation of the target. The relative movement between respective frames of the image data is automatically estimated by a motion estimator, based on frames of data from the tracking arrays. As the transducer is rotated about the azimuthal axis of the image data array, features of the target remain within the image planes of the tracking arrays. Movements of these features in the image planes of the tracking arrays are used to estimate motion as required for the three-dimensional reconstruction. Similar techniques estimate motion within the plane of an image to create an extended field of view.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a division of application Ser. No. 09/340,542, filedJun. 28, 1999, which is now U.S. Pat. No. 6,201,900 issued Mar. 13,2001, which is a divisional of (Ser. No. 08/916,585), now U.S. Pat. No.6,014,473, which is a CIP of (Ser. No. 08/807,498, filed Feb. 27, 1997now abandoned); which is a CIP of (Ser. No. 08/621,561, filed Mar. 25,1996 now abandoned).

This application is a continuation-in-part of U.S. patent applicationSer. No. 08/621,561, filed Mar. 25, 1996, now abandoned which is in turna continuation-in-part of provisional U.S. patent application Ser. No.60/012,578 filed Feb. 29, 1996, and assigned to the assignee of thepresent invention.

BACKGROUND OF THE INVENTION

This invention relates to an improved system, method and transducer foracquiring two-dimensional image information and relative positionalinformation regarding the image information to allow subsequentthree-dimensional or extended field of view reconstruction.

There is growing interest in three-dimensional ultrasonic images. Oneapproach is to use a two-dimensional transducer array to obtainthree-dimensional image information directly. A two-dimensional arraycan be used to scan electronically in any desired orientation, andthereby to acquire the desired information. This approach brings with itconsiderable problems related to fabrication difficulties, signal tonoise difficulties and processing difficulties.

Another approach is to collect multiple two-dimensional image dataframes using a one-dimensional transducer array along with relativepositional information among the image data frames so that these framesmay be subsequently assembled in a three-dimensional volume to form thedesired three-dimensional reconstruction. One approach is to use amotorized array to collect the desired set of image data frames byprecisely controlling the movement of the transducer array. One exampleis the transducer shown in U.S. patent application Ser. No. 08/267,318(Hossack, et al., assigned to the assignee of the present invention).See also Pini U.S. Pat. No. 5,159,931). A related approach is to use alarge rotating transducer as described in McCann et al.,“Multidimensional Ultrasonic Imaging for Cardiology” (Proceedings ofIEEE, 76, 9, pp. 1063-1072, September 1988). Another approach is to usemanual motion detection techniques based on analysis of ultrasonicimages. See Sapoznikov et al., “Left Ventricular Shape, Wall Thicknessand Function Based on Three-Dimensional Reconstruction Echocardiography”(“Computers in Cardiology,” IEEE Computer Society Press, Cat CH 2476-0,pp. 495-498, 1987); and Taruma et al., “Three-Dimensional Reconstructionof Echocardiograms Based on Orthogonal Sections” (Pattern Recognition,18, 2, pp. 115-124, 1985). Manual techniques are slow and cumbersomeand, therefore have many drawbacks.

Schwartz U.S. Pat. No. 5,474,073 describes a qualitativethree-dimensional method using a hand-held transducer array and anassumed scan motion. Qualitative methods have drawbacks, and the presentinvention is directed to a quantitative method.

Keller U.S. Pat. No. 5,353,354 discloses a transducer array equippedwith accelerometers or magnetic sensors designed to measure theorientation of the transducer, and therefore relative motion betweenrespective image planes. Once the image plane positional information isprovided, standard methods are employed for assembling the image planeinformation into a three-dimensional volume and providing an appropriatedisplay such as a cross section, a surface rendering, a segmentation, orthe like. One drawback of the Keller approach is that it relies onassessing the position of the transducer array with respect to a fixedsurface exterior to the patient, not with respect to the patient orother target. If the patient moves, the absolute position of all targetregions is moved, and the accuracy of the three-dimensionalreconstruction is degraded or eliminated entirely. Magnetic sensorsystems have additional disadvantages, such as potential vulnerabilityto local magnetic interference from metal objects and electromagneticfields generated by electronic devices such as cathode ray tubes.Accelerometer systems can be bulky and susceptible to cumulative error,since they rely on two stages of integration to convert accelerationinformation to displacement information.

The present invention is directed to a new system and transducer whichto a large extent overcome the problems discussed above.

SUMMARY OF THE INVENTION

According to a first aspect of this invention, a method is provided forregistering image information acquired from a target, comprising thefollowing steps:

(a) acquiring a plurality of sets of image information with at least oneultrasonic transducer array, said at least one array moved between atleast some of the sets of image information, said plurality of setscomprising a plurality of image data sets and a plurality of trackingsets;

(b) automatically determining a component of motion based on acomparison of at least a portion of the tracking sets acquired in step(a); and

(c) automatically using the component of motion determined in step (b)to register selected ones of the image data sets acquired in step (a).

According to one aspect of this invention, the image data sets areacquired with different transducer arrays. According to another aspectof this invention, motion estimates are used to create an extended fieldof view, and in this case no separate array may be required for thetracking sets.

This invention is also directed to an improved transducer suitable foruse in the method described below. The transducer of this inventionincludes a support element and first, second, and third transducerarrays coupled to move with the support element. The first transducerarray includes first transducer elements arranged along an azimuthalaxis and having first and second ends spaced along the azimuthal axisand a first central image plane. The second transducer array comprisessecond transducer elements positioned near the first end of the firsttransducer array and comprising a second central image plane. The thirdtransducer array comprises third transducer elements positioned near thesecond end of the first transducer array and comprising a third centralimage plane. The first and second central image planes are non-parallel,as are the first and third central image planes.

This arrangement for an ultrasonic transducer allows both trackinginformation and image information to be collected concurrently. Thetracking information is used to determine estimates of the movement ofthe transducer and/or the target between respective image data frames.This information can then be used in registering the respective imagedata frames appropriately in three-dimensions to form the desiredthree-dimensional representation.

Further aspects and advantages of the invention are discussed below inconjunction with the preferred embodiments.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an ultrasonic imaging system thatincorporates a presently preferred embodiment of this invention.

FIG. 2 is a schematic perspective view of the transducer of FIG. 1.

FIG. 3 is a schematic side view of the transducer of FIG. 2.

FIG. 4 is a schematic plan view of the transducer of FIG. 2.

FIG. 5 is a schematic view of a portion of the system of FIG. 1.

FIG. 6 is a schematic perspective view showing the movement of imageregions as the transducer of FIG. 2 is rotated about the azimuthal axis.

FIGS. 7, 8, 9 and 10 are schematic views showing images acquired withthe transducer of FIG. 6 at four rotational positions of the transducer.

FIG. 11 is a schematic perspective view showing the operation of thetransducer of FIG. 2.

FIG. 12 is a schematic plan view of a tracking image region generatedwith the system of FIG. 1.

FIG. 13 is a schematic view showing the averaging of estimates of motionin the system of FIG. 1.

FIG. 14 is a schematic view showing how the bulk motion of an imagedorgan can be recognized and disregarded in the system of FIG. 1.

FIG. 15 is a schematic view showing the movement of the target amongacoustic lines as the transducer of FIG. 2 is rotated about theazimuthal axis.

FIG. 16 is another view of the raw acoustic line data of FIG. 15.

FIGS. 17, 18 and 19 are three schematic perspective views showing themanner in which multiple image data frames can be registered withrespect to one another in three-dimensions to form a three-dimensionalrepresentation.

FIG. 20 is a view of a display generated by the system of FIG. 1.

FIG. 21 is a schematic view of a modified version of the transducer ofFIG. 1.

FIG. 22 is a schematic view of an alternative ultrasonic imaging system.

FIGS. 23 through 26 are plan views of transducers using crossed arrayssuitable for use in selected embodiments of this invention.

FIGS. 27, 27 a through 27 h, and 27 j are plan views of alternativetransducers suitable for use in selected embodiments of this invention.

FIG. 27i is an end view taken along line 27 i—27 i of FIG. 27h.

FIGS. 28 and 29 are plan views of other transducers suitable for use inselected embodiments of this invention.

FIG. 30 is a block diagram of an ultrasonic imaging system whichincorporates a preferred embodiment of this invention.

FIGS. 31 and 32 are flowcharts of routines performed by the imagingsystem of FIG. 30.

FIG. 33 is a schematic diagram illustrating operation of an alternativeembodiment.

FIGS. 34 through 37 are graphs illustrating operation of a fuzzy logicsystem included in an alternative form of this invention.

FIG. 38 is a diagram showing the manner in which two frames can beweighted to form an extended field of view.

FIGS. 39 and 40 are schematic views that relate to a single block motionestimation system.

FIGS. 41 and 42 are schematic views of alternative systems forgenerating transmit signals.

FIG. 43 is a schematic view of a system for detecting an erroneousmotion estimate.

DETAILED DESCRIPTION OF THE PRESENTLY PREFERRED EMBODIMENTS

Turning now to the drawings, FIG. 1 is a block diagram of an ultrasonicimaging system 10 which incorporates a presently preferred embodiment ofthis invention. The following discussion will first present a systemoverview, and then a detailed description of selected components of thesystem.

System Overview

The system 10 includes a beamformer system/signal detector 12 whichincludes both transmit and receive beamformers and is connected via amultiplexer/demultiplexer 14 to an ultrasonic transducer 16. Anysuitable devices, including conventional devices, can readily be adaptedfor use as the elements 12, 14.

The ultrasonic transducer 16 will be described in greater detail belowin conjunction with FIGS. 2-4. Here, it is important to note that thetransducer 16 preferably includes three separate transducer arrays 18,20, 22. The array 18 is used for collecting image data that will be usedto construct displayed representations of the target. The arrays 20, 22are smaller arrays oriented in this embodiment at right angles to theimage data array 18 to operate as tracking arrays. The tracking arrays20, 22 are used in this embodiment to estimate the motion betweenrespective image data frames from the image data array to allow theimage data frames to be registered properly for reconstruction.

The beamformer system/signal detector 12 sends excitation signal pulsesto the arrays 18, 20 and 22 and supplies summed returning echoes to asignal detector. The output of the signal detector is supplied to a scanconverter 24. The beamformer system/signal detector 12 operates thearrays 18, 20, 22 in the conventional manner as phased arrays byproperly timing the excitation signals applied to the arrays 18, 20, 22,and by properly timing and phasing the received signals prior tosumming. The scan converter 24 controls an output display 26 to displaypreferably the three images generated by the three arrays 18, 20, 22along with additional information as described below.

In addition, scan-converted image information from the scan converter 24is stored in a data storage system 28. In this embodiment the datastorage system 28 includes three separate storage arrays, each storingdata for image frames from a respective one of the arrays 18, 20, 22.Thus, image information from the image data transducer array 18 isstored as frames of image data in the storage array 30, and imageinformation from the tracking transducer arrays 20, 22 is stored asrespective frames of image data in the storage arrays 32, 34,respectively. The frames of data in the storage arrays 30, 32, 34 areall time marked, so that they can be associated with one anotherappropriately. This time marking can take the form of real-time clockinformation or frame number information, for example.

The frames of image data in the storage array 30 are applied to acomputer 36. It is these frames that are used to form the displayedrepresentation of the target. The tracking image frames stored instorage arrays 32 and 34 are not registered to create a displayedreconstruction of the target, but are instead used to determine therelative positions of individual frames of image data from the imagedata storage array 30.

In order to estimate movement of the target between successive frames ofthe image data, the image information from the tracking array datastorage arrays 32, 34 is supplied to a motion estimator 38. The motionestimator 38 compares sequences of images from the tracking transducerarray 20 and the tracking transducer array 22 to estimate a component ofmotion of the transducer 16 between the respective frames. This estimateof the component of motion is smoothed in logic 40, and then applied toa calculator 42 that calculates a vector value defining the bestestimate of the movement between selected frames of the image datastored in the image data storage array 30. This vector is then appliedas another input to the computer 36.

In the system 10, the elements 28 through 42 can be designed to operatein real time, and the motion vectors can be displayed on the outputdisplay 26 as discussed in conjunction with FIG. 20. Once a full set ofdata has been acquired, the image data frames and the frame-to-frametranslation vectors can then be transmitted to the specialized computer36 which can either be combined with or external to the ultrasonicimaging system.

The computer 36 registers selected frames of image data from the imagedata storage array 30 with respect to one another by appropriate use ofthe vectors supplied by the calculator 42. Also any necessaryinterpolation is done, and the respective frames of image data arestored in proper registration with respect to one another in athree-dimensional data storage device 44. The computer 36, whenoperating in a display mode, can select appropriate information from thethree-dimensional data storage device 44 to provide a desired image onthe display 46. For example, cross sections can be taken in variousplanes, including a wide variety of planes that do not correspond to theplanes of the image data. Also, surface renderings and segmentationdisplays can be created if desired.

In one mode of operation, the transducer 16 is rotated through a sweepunder the direct manual control of an operator, smoothly fromside-to-side about a single axis of rotation lying along the azimuthalaxis on the face of the image data array 18. The method described belowcan account for imperfections in the sweep. During three-dimensionalreconstruction, the quality of the reconstruction degrades gracefully asa result of positional error. Distortion rather than blurriness is theresult of imperfect motion detection.

The Ultrasonic Transducer 16

FIGS. 2-4 provide three views of the ultrasonic transducer 16. As shownin FIG. 4, the three arrays 18, 20, 22 each comprise a respective set oftransducer elements 48, 50, 52, all mounted on a common support element53. The transducer elements 48 are arranged along an azimuthal axis A,and the image data transducer array 18 defines first and second ends 54,56. The tracking arrays 20, 22 are each positioned near a respective oneof the ends 54, 56, centered on the azimuthal axis A. The transducerelements 50, 52 are arranged along respective tracking axes T1, T2, andthe tracking axes T1, T2 are in this preferred embodiment substantiallyperpendicular to the azimuthal axis A. The tracking arrays 20, 22 areeach shorter than the image data array 18, and each has fewer transducerelements 50, 52. Each of the transducer elements 48 can be spaced at 1/Ntimes the pitch of the transducer elements 50, 52. As shown in FIG. 3,the tracking arrays 20, 22 can be oriented to point inwardly, toward theimage data array 18. Alternately, the tracking arrays 20, 22 can becoplanar with the array 18, to provide a preferred profile for thetransducer 16.

The image data array 18 can be of conventional form, such as a flatlinear array with a cylindrical elevation focusing lens. Alternately,the array can be generally flat, but the transducer elements can becurved in elevation to focus. In this case a non-refractive filler suchas a polyurethane can be used since a focusing lens is no longerrequired. Whether or not a lens is used, the image data array may becurved in azimuth to yield a larger field of view. The tracking arrays20, 22 will typically include a lens to achieve the desired focus inelevation. Since the curvatures of the various lenses or arrays will bein differing planes, a non-refractive filler section may be formed onthe transducer 16 to yield the preferred smooth shape. Alternately, thetracking arrays 20, 22 may also be curved with non-refractive windowsformed on top of the desired shape. Both the image data array 18 and thetracking arrays 20, 22 may be phased sector, Vector®, linear orcurvilinear. All imaging modes including B mode, color Doppler, colorDoppler energy and the like are supported. A conventional TEE transducersuch as the biplane V510B transducer of Acuson can be used in atwo-transducer embodiment.

The transducer geometry shown in FIGS. 3 and 4 can be used to obtainimage planes as shown in FIG. 2. The image plane 58 of the image dataarray 18 in this embodiment passes through the azimuthal axis A. Imagedata collected with the image data array 18 is positioned along scanlines 64 in the image plane 58. The image planes 60, 62 of thetransducer arrays 20, 22, respectively, are oriented transversely to theimage plane 58. The image planes 58, 60, 62 are the central image planesof the respective arrays, that is the only image plane for a 1D arrayand the central plane (i.e. the plane not steered in elevation) for a1.5D array.

For the sake of simplicity, the tracking arrays 20, 22 may haveidentical transducer element pitches to that used by the image dataarray 18. This approach allows the same beamformer delays to be used forall three arrays 18, 20, 22. However, in many applications the trackingarrays 20, 22 are adapted to form relatively fewer acoustic lines. Thisis particularly the case if motion detection is concentrated in thevicinity of the center line of the image planes 60, 62. If only a narrowfield of view is required for the tracking arrays 20, 22 then thetracking array pitch may be coarser, for example twice the pitch of theimage data array 18.

By making the tracking array pitch an integer multiple of the image dataarray pitch, the same beamforming delays can be used, but with theappropriate channels disconnected, as shown in FIG. 5. In FIG. 5 theelement-to-element pitch of the tracking array 20 is twice that of theimage data array 18, and consecutive transducer elements 50 of thetracking array 20 are connected to only the even or odd signal lines forthe transducer elements 48 of the image data array 18. In the limit,each tracking array 20, 22 may be composed of as few as two transducerelements, although this will limit the maximum resolution that isachievable.

Common signal conductors can be used between the beamformer/signaldetector 12 and the housing for the transducer 16. In the housing,individual signals are routed between the signal conductors and thetransducer elements 48, 50, 52 by high voltage analog switches ormultiplexers, such as those available from Supertex Inc., Sunnyvale,Calif. and having the family designation HV2xx.

FIGS. 6-11 are schematic views that illustrate the manner in whichimages generated by the tracking arrays 20, 22 can be used to estimatethe movement between images generated with the image data array 18. FIG.6 shows a perspective view for three separate positions of thetransducer 16. These three separate positions of the transducer 16 areobtained by rotating the transducer 16 about the azimuthal axis. As theimage data array 18 rotates about the azimuthal axis A, the image planes58A, 58B, 58C rotate in a fan-like manner, Thus, each of the imageplanes 58A, 58B, 58C of FIG. 6 is disposed in a separate, respectiveplane of three-dimensional space.

In contrast, the image planes 60A, 60B, 60C; 62A, 62B, 62C for eachtracking array 20, 22 remain coplanar as the transducer 16 is rotatedabout the azimuthal axis A. The actually imaged regions within the imageplanes 60A, 60B, 60C; 62A, 62B, 62C rotate about the azimuthal axis A asthe transducer 16 rotates. In many applications, the imaged regionswithin the image planes 58A, 58B, 58C will not overlap or intersect theimaged regions within the image planes 60A, 60B, 60C or the imagedregions within the image planes 62A, 62B, 62C. This arrangement canreduce cross talk and other interference problems, as discussed below.

FIGS. 7-10 show four sets of images. Each set includes an image 66A, B,C, D from the image data array 18 and a corresponding image 68A, B, C, Dfrom one of the tracking arrays 20, 22. In this case the target is asphere, and the images 66, 68 intersect such that the sphere appears inboth images 66, 68. As shown in images 66A, B, C, and D, various crosssections of the sphere are displayed as the transducer 16 rotates aboutthe azimuthal axis. The cross sections shown in images 66A and 66D showsmaller diameter disks taken near an edge of the sphere, and the images66B and 66C show larger diameter disks taken near the center of thesphere. Thus, the disks shown on the images 66A, B, C and D differ indiameter, in accordance with the moving plane of the image (see FIG. 6).In contrast, the images 68A, B, C, and D all show disks of the samesize. Because the plane of the images 66A, B, C, and D remains the same,as discussed above in conjunction with FIG. 6, the disk that isdisplayed in these images remains constant in size but moves across theimage plane. The location of the disk as it moves from one image to thenext provides a measure of a component of motion of the transducer 16 inthe image plane of the images 68A, B, C, D.

If the image plane of the transducer arrays 20, 22 are not perpendicularto the surface of the image data array 18 (for example because thetracking arrays 20, 22 are pointed inwardly as shown in FIG. 3), it maybe preferred to use a cosine Θ correction factor to take account of thedifference between image range and physical depth perpendicular to theimage data array 18.

An important advantage of the transducer 16 is that it includes twotracking arrays 20, 22, each positioned near an adjacent end 54, 56 ofthe image data array 18. This arrangement allows compound rotations tobe assessed. In FIG. 11 the transducer 16 is rotated about a rotationalaxis RA oriented as shown. In FIG. 11 the solid-line circles denote theimage of the target at a first point in time, and the dotted-linecircles denote the image of the target at a subsequent point in time,after rotation about the rotational axis RA. Note that the images movein opposite directions in the image planes 60, 62 in this situation. Bycomparing the component of motion determined separately from the twotracking arrays 20, 22 the actual rotation of the target with respect tothe transducer 16 can be determined.

If desired, the transducer 18 can include an absolute sensor forposition, orientation, or both, such as a magnetic sensor 19 as shown inFIG. 21 or an accelerometer. The sensor 19 may be used to supplement orback up the motion detection approach described in detail below, and maybe of the types described in Keller U.S. Pat. No. 5,353,354.

The Beamformer System/Signal Detector 12

As the transducer 16 is rotated substantially about the azimuthal axisA, the image data array 18 and the tracking arrays 20, 22 aresequentially operated. When the arrays 18, 20, 22 are individuallyaddressable, the transmit beamformer in element 12 sends appropriatelytimed and phased excitation signals to the respective transducerelements 48, 50, 52. In one embodiment a frame of data from the imagedata array 18 is collected, and then frames of data from the trackingarrays 20, 22 are collected. Alternately, the transmit beamformer mayalternate between the image data array 18 and the tracking arrays 20, 22between individual scan lines or between groups of scan lines. If theimage planes of the arrays 18, 20, 22 are non-intersecting, andparticularly if the image data array 18 is excited at a differentfrequency than the tracking arrays, then the risk of cross-arrayinterference is small. For this reason the pulse repetition rate (whichis normally limited by the time required for ultrasonic signals toattenuate in the body between successive scan lines) may be increased.

In some applications the arrays 18, 20, 22 may be connected in common tothe transmit beamformer. Typically, the imaging requirements for theimage data array 18 will differ substantially from those for thetracking arrays 20, 22. Image data quality should be maintained at ahigh level, but tracking data need only be of sufficient quality topermit reliable tracking. Costs may be reduced by sharing some cableconductors between the array 18 and the arrays 20, 22. Typically,elements near the ends of the image data array 18 are most suitable forsharing, since they are less important than center elements.

In order to reduce interference between the arrays 18, 20, 22 cablerouting from the elements 48 of the image data array 18 to the elements50, 52 of the tracking arrays 20, 22 is preferably jumbled in a knownpseudo-random manner. The element 12 uses the jumbling scheme to sortthe beamformer delays such that one set of arrays (either the image dataarray 18 or the tracking arrays 20, 22) is operated coherently at anygiven scan line. The other set is operated incoherently because of thecable jumbling. The optimum jumbling scheme may be determined by routineexperimentation.

Cross talk may be further reduced with frequency coding and voltagelevel techniques. The tracking arrays 20, 22 may operate with a reducedimage depth, such as a few centimeters, and therefore a high frequencysuch as 10 MHz. The image data array 18 may operate at a longer range,lower frequency such as 5 MHz, thereby reducing cross talk. Thetransducer elements 48, 50, 52 may be formed with thicknesses thatselect the appropriate frequencies. Also, bandpass filters may be usedin the element 12 to select the desired frequency bands for detection.

Voltage levels may vary between the two sets of arrays 18, 20, 22. Forexample a higher voltage may be used when the tracking arrays 20, 22 areselected, particularly if the tracking arrays 20, 22 require a highervoltage to operate effectively. In this case the tracking arrays 20, 22emit a relatively small signal when the image data array 18 is selectedand the voltage level is reduced.

FIG. 22 shows an alternative system 10′ which uses many of the samecomponents as the system 10. The system 10′ differs in that the arrays18, 20, 22 are operated simultaneously at different frequencies bybeamformer/signal detectors 12′, 12″. In each case a respective bandpassfilter 13′, 13″ is provided to isolate the bandpass of interest. Asdescribed above, the tracking arrays 20, 22 may operate at 10 MHz andthe image data array 18 may operate at 5 MHz. The jumbled cable routingdiscussed above may be used to reduce interference. The bandpass filters13′, 13″ can operate on beamformed signals, but the arrangement of FIG.22 is preferred in practice.

When the transducer 16 is swept by rotating it about the azimuthal axisA, the preferred format for the image data array 18 is the sectorformat. Acoustic line data acquired by in the sector format can beconveniently used in the correlation process without scan conversion,since, for example, a pure rotation in rectangular coordinates can berepresented as a change in an angular coordinate in a suitably chosencylindrical coordinate system, given knowledge of the line to lineangular increment, the angles of the lines with respect to the normalline (see FIG. 15), and the sample to sample range increment.

The Motion Estimator 38

Motion detection may be performed manually, for example by placing aline on the display data at a particular recognizable feature in theimage planes 60, 62 and then repeating this activity on subsequentframes. The system can keep track of the line position for successiveframes to generate a vector indicative of frame-to-frame motion.

A better method is to use computer analysis of frame-to-frame motionusing a cross correlation or similar method on the image data acquiredwith the tracking arrays 20, 22. Such techniques (which will be referredto herein generally as correlation techniques) have been used in thepast for tracking blood flow. These methods do not require that arecognizable feature be present in the display area, and they canfunction adequately using only ultrasound speckle data. Speckle is anaturally occurring phenomenon in ultrasound images and is a result ofthe coherent nature of the reflected waves from small scattering objectsin the tissue.

Any suitable correlation technique can be used, including crosscorrelation and the sum of absolute differences method as discussed inBohs and Trahey “A Novel Method For Angle Independent Ultrasonic ImagingOf Blood Flow And Tissue Motion” (IEEE Trans. on Biomed. Eng., 38, 3,pp. 280-286, March, 1991). Cross correlation is the well-knownmathematical operation which uses sequentially obtained sums ofmultiplication operations for various translations of data in a searchfor the translation for which the two sets of data are best matched. Thesum of absolute differences method is a computationally simplercorrelation technique, but it achieves a similar net effect. The sets ofdata are translated by varying amounts. For each translation respectivedata values in each of the sets are subtracted and the sum of theabsolute differences is calculated. If a particular translation gives asum of absolute differences that is close to zero, then it is probablethat the sets of data have been aligned by the associated translation.This translation required to achieve an alignment is an indication ofthe motion between the two respective frames at the sides of the imageclosest to the respective tracking array. As explained below, the motionat other parts of the image can be evaluated using the detected motionat both tracking arrays and linear interpolation techniques along theazimuth of the image data array 18.

The size of the data block used in either type of correlation is amatter for optimization. Bigger blocks will have a reduced likelihood ofa false match but will require longer calculation times. The maximumframe-to-frame displacement that can be detected is limited by the blocksize. Typically, searches are made to plus or minus one-half of a blocksize. By way of example, a 16×16 pixel block may be used to detect amaximum translation of plus or minus 8 pixels.

The motion estimator 38 can use any effective technique to determinemotion based on the frames of data stored in the arrays 30, 32, 34.Motion estimation may be based on the entire frame or on portions of theframe. When portions are used, they may be selected to correspond to awell-defined feature in the image. Motion estimation may be used on aspaced subset of available data to save time if that is more efficient.For example, if samples are available on a 0.5 mm spacing, but optimizedmotion detection can be performed based on a 1 mm spacing, time can besaved by deleting alternate samples and performing motion detectionbased on only a subset of the available data.

The tracking image data used for the motion estimation may take any oneof a number of forms, including at least the following:

1. Adjacent pixels in the output display (which is interpolated from theactual acoustic line data);

2. Selected pixels from a larger grid, such as every Nth pixel;

3. Averages of groups of pixels, such as the average of a group of fouradjacent pixels;

4. Samples from pre-scan conversion beamformer acoustic line data.

The beamformer outputs lines of acoustic data may be in polar orCartesian format. Since the relation between these lines of data and thephysical lines of propagation are known, samples derived from theseacoustic lines may be used in the motion detection.

FIG. 15 is a schematic view showing the position of a target at frames Nand N+1 with respect to acoustic scan line data. In FIG. 15 referencesymbol 70 is used for individual points at which the measurements weretaken. These points are arranged at discrete intervals along the scanlines. FIG. 16 shows the raw acoustic line data (prior to scanconversion), in which each row represents a respective scan line, andthe points of measurement at respective ranges are as illustrated. InFIGS. 15 and 16 the + symbol is used for the position of the target forboth frames N and N+1. The motion estimator 38, when operating on framesN and N+1, detects that the target has moved from scan line 1 to scanline 3 and the range has remained constant at four sample intervals fromthe origin. Since the scan lines are spaced at an angular separation ofX degrees and the distance between adjacent points 70 at the center ofthe motion block is equal to P millimeters, basic geometricalrelationships can be used to prove that the lateral offset equals+8PsinXsinΘ and the depth offset equals −8PsinXcosΘ, where Θ is theangle in degrees between the center of the motion detection block andthe normal line emanating from the scan line origin, and where thetracking array is operating in sector format. These calculations areapproximate, but they demonstrate the manner in which raw acoustic linedata can be used in the motion estimator 38.

When acoustic line data is used in the motion estimator, it cancorrespond to digitized RF, digitized IF, digitized baseband, orrectified low-pass-filtered, envelope-detected signals. As an example ofenvelope-detected signals, B-mode data can be used. The acoustic linedata signals may be real digital samples or complex (I, Q) data samples.

In the presently preferred embodiment the approach used in the motionestimator 38 relies on the sum of absolute differences in a pixel blockequal to 16×16 pixels. For reasons of efficiency it is convenient to usea specialized integrated circuit. The LSI Logic 64720 integrated circuitis designed to perform 16×16 motion detection at rates of up to 396blocks in {fraction (1/30)}th of a second. These circuits may becombined to yield higher throughput or larger block sizes. If the blocksize is reduced to 8×8 pixels, still higher rates are possible (4500blocks in {fraction (1/30)}th of a second). This integrated circuit hasthe cost benefits of volume manufacture. Similar alternative integratedcircuits may also be used. Alternatively, the entire operation can beperformed using a suitably programmed general purpose processor such asthe Texas Instruments TMS 320C80.

Preferably, the arrays 18, 20 are aligned with the array 16 such thatthe image plane 58 is aligned with the vertical axis of the image planes60, 62. For this reason, motion with respect to the centerline of theimage planes 60, 62 can be used to determine the relative motion of theimage plane 58. Data blocks are defined lying along the center line.

Strictly only two motion detection operations per tracking array arerequired. However, by performing multiple tracking operations overvarious parts of the image (not necessarily on the center line), greaterconfidence in the accuracy of the method can be obtained. For example,as shown in FIG. 13 a smooth, straight line fit can be used to obtain anaverage of multiple estimates of the component of motion being detected.In FIG. 13 the dashed arrows represent the fitted component of motionand the continuous arrows represent the actual measures of the componentof motion. By taking an average of multiple measurements a more reliableestimate is obtained. This average can be used both with respect to theamplitude and to the angle of the motion component vectors.

Also, motion estimates which are radically different from neighboringestimates may be discarded as inaccurate. As shown in FIG. 14 an imageof one of the tracking arrays can include an organ 72 having a bulkmotion indicated by the arrows 74 which is different from the motionindicated by the speckle of the larger part of the area of the image. Inthis case the bulk motion of the organ can be disregarded, and theaverage of the remaining estimates of motion indicated by the shorterarrows 76 can be used as an estimate of the motion of the frame. Onesuitable approach is to quantize the motion vectors (length anddirection), and then to find the most commonly occurring quantizedlength and direction. Next the actual (unquantized) vectors within atolerance band (e.g. ±30%) of the most commonly occurring values areselected and averaged to generate the desired estimate,

Additionally, in a sum of absolute differences calculation, the ratio ofthe minimum sum to the average sum (after removing the minimum sumvalue) can be used as an indicator of the quality of the image motiondetection. If the minimum sum is close to the average, then thecalculation is susceptible to error, i.e. falsely detected motion. Whenthe LSI 64720 integrated circuit is used, the sum of the error valuesoutput is an indicator of a low quality result (a high error value sumcorresponds to a low quality result).

Low quality tracking may also be detected by comparing estimates ofmotion. Adjacent or successive estimates of motion which are not similarto one another may be an indication of low quality motion estimation. Ofcourse, two or more of the approaches described above may be used incombination.

Speckle patterns change rapidly as motion occurs. In particular, aspeckle pattern will change if the elevation slice position moves. Thisis because speckle scatterers move in and out of the image plane ratherthan remaining within the plane and moving parallel to it. For thisreason it is preferable in many cases to make frequent motion estimatesas the transducer 16 is moved. The exact frequency of the estimates willdepend upon the scale of the motion detection problem, which is relatedto the speed of motion imposed by the operator, the operator's abilityto maintain slice position, and the nature of the speckle target (whichis a function of tissue type and ultrasound frequency).

By way of example, it may be determined that thirty image data framesfrom the image data array 18 provide sufficient imaging data for adesired 3-D reconstruction. However, it may be preferable to performfive motion estimates between each selected image data frame and to sumthese estimates to calculate the net motion between the image dataframes. Since cumulative errors are to be minimized, one way to achievethis result is to over-sample the motion detection operation and toaverage the results to reduce quantitation effects due to pixel size.With this approach there might be for example five frames from each ofthe tracking arrays 20, 22 for each frame of the image data array 18.

While motion detection often works best when performed from a sum ofsmall motion detections, there are situations when it is of value to usemotion detection between frame N and frame N-M, where M is a integergreater than 1, such as 10. In principle, this approach can be used tocorrect cumulative errors. However, the probability of such a largemotion being correctly estimated is less. Therefore, the value of M is amatter for optimization.

Since elevation slice affects speckle quality and stability, it may bepreferable to track motion at the elevation focus of the lens.Alternately, if the speckle dependence is too sensitive at the elevationfocus, it may be preferable to avoid that area. It may be preferable toapply varying weightings to the detected motions during the smoothingoperation to maximize the net accuracy by taking more account of moreaccurate data. It may be preferable to use an elevation focused array(1.5D array) or perhaps an unfocused array if that appears duringexperimentation to provide the best results.

The data used for motion detection can taken many forms.Envelope-detected, scan converted data is a simple design choice, and ispresently preferred. Alternatives include envelope-detected data priorto scan conversion and RF or baseband beamformer output signals prior toenvelope detection.

It is also possible for the user to select regions of interest if forsome reason the user has specific preferences about which regions in thetracking data image would provide the best motion detection. Forexample, a user may choose not to use volumes of moving blood as regionsfor motion detection.

During the collection of image data the preferred display is as shown inFIG. 20. In FIG. 20 an image from the image data array 18 is showncentrally on the screen and tracking images from the tracking arrays 20,22 are shown on respective sides of the screen. Preferably, motiondetection is performed in real time, and the detected motion ispresented on the display by indicating the calculated motion as itoccurs in real time. This display can take the form of motion vectors asshown in FIG. 20. Alternatively numeric measures of detected motion canbe displayed. These displays indicate the relative position of thetransducer in the sweep, and therefore provide the operator with anindication as to when the desired angular motion has been completed andalso provide an indication of whether the motion detection system isworking properly. Alternately, the image data may be displayed withoutthe tracking images or without display of the detected motion. In theevent that the system-calculated motion vectors indicate low-qualitytracking (such as erratic changes in the calculated motion vector withina single frame or between successive frames) the system can prompt theoperator to begin again, as for example with an audio prompt such as analarm or a visual prompt such as a flashing, reverse video message.

If desired, the system 10 can be programmed to assist the user inachieving an optimum sweep rate. In many cases, optimum use of themotion estimators calls for an optimized sweep rate, i.e., a sweep ratethat is large enough to avoid unacceptable error accumulation and smallenough to remain within the range of movement of the motion estimator.Either an audio or a video prompt may be used,

For example, when the motion estimator is adapted to detect ±8 pixelmotions, the system may emit an intermittent tone when the sweep rate isapproximately correct (e.g., 4 pixels of movement per estimate). If theestimated movements are too small (e.g., less than 2 pixels), theintermittent tone is replaced with a low continuous tone, which becomeseven lower if the sweep rate slows further. Conversely, if the estimatedmovements are too large (e.g., greater than 6 pixels), the intermittenttone is replaced with a high continuous tone, which becomes even higherif the sweep rate speeds up.

A suitable visual prompt includes a variable-length arrow, which canhave a longer length to prompt a faster sweep rate and a shorter lengthto prompt a slower sweep rate, and which can flash to indicate anoptimum sweep rate.

Another approach is to program the motion estimator to select thespacing in time of the frames that are correlated to estimate thecomponent of motion. By properly selecting this spacing in an adaptivemanner, the measured component of motion can be kept within the optimumrange for a wide range of sweep velocities. For example, if there aremany non-used tracking frames between each pair of correlated trackingframes, and if 8 pixels is the maximum detectable motion, the number ofnon-used tracking frames between each pair of correlated tracking framescan be increased or decreased as necessary in real time to maintain thedetected motion in the range of 4 to 6 pixels.

Low quality motion estimates can be weighted at a low level or entirelyremoved. One approach for selecting low quality motion estimates isfirst to fit a curve to all of the motion estimates (both in length andangle). Then the individual motion estimates are compared to the fittedcurve, and motion estimates that deviate from the curve by more than athreshold amount such as 30% are considered low quality motion estimatesand are deleted from the collection of motion estimates. Then the curvefitting operation is repeated using only the remaining motion estimates.If more than a certain fraction such as 20% of the motion estimates areclassified as low quality estimates, the operation can be abandoned andthe user prompted to repeat the sweep.

In general, it is often necessary only to determine the component ofmotion relating the relative positions of successive images. Ifrequired, it is also possible to determine the axis of rotation bydrawing lines perpendicular to the determined motion vectors, as shownin FIG. 12. When this is done the length of the motion vector is anindication of the distance of the motion vector from the axis ofrotation.

Strictly speaking, it is not necessary to have two tracking arrays 20,22. However, when two tracking arrays are used, the ability to deal withimpure rotation (where one end of the image data array 18 is rotatedmore than the other) is substantially increased. Furthermore, whenmultiple tracking arrays are used and motions for various ranges on eachtracking array have been determined, interpolation may be applied tofind the center of rotation that applies for all intermediate points inthe range direction on each tracking array.

Since the tracking arrays are on either side of the image data plane,and the exact geometry of the image data plane with respect to thetracking arrays is known, it is possible to interpolate linearly alongthe image data array azimuth axis to calculate the exact pixeltranslations for all points on the image data plane.

Typically, motion estimates are collected along a straight line oracross a rectangular grid. Due to the constraints of the array geometryand the propagation of the straight acoustic lines from the array, thetheoretical behavior of the motion vectors as a function of depth mustsatisfy certain constraints. In particular, the lengths of the motionvectors should vary linearly with depth. These constraints can be usedto reduce the error in the estimated motion. For example, a sequence ofmotion estimates can be acquired as a function of depth and thenconverted to motion estimate vectors (length and angle). A straight lineis then fitted using well known methods to determine the best fittingline to the actual data for the length component. A second straight lineis then fitted to the actual data for the angle or direction component.These fitted lines (comprising length and direction) can be linearlyinterpolated along the azimuthal direction during three-dimensionalreconstruction at intermediate locations other than those used to derivethe motion vectors.

Note that if the patient and the transducer 16 move slowly together,loss of performance is not necessarily the result, as long as therelative position of the transducer 16 with respect to the patient islargely maintained.

In order further to improve motion estimation it may be desirable toutilize only images corresponding to selected portions of the ECG cycleor the breathing cycle. Both ECG gating and breathing gating are wellknown in three-dimensional reconstruction of images. See, for example,McCann et al. “Multidimensional Ultrasonic Imaging for Cardiology” at p.1065. With ECG gating a window is selected a fixed time duration afterthe ECG pulse maximum. With breathing gating it is often simplest to askthe patient to hold his or her breath for the short duration of theultrasonic scan. Alternatively, chest motion can be recorded using adisplacement sensor, and data can be selected for a portion of thebreathing cycle.

Various other techniques can be used to optimize motion estimation. Forexample, accuracy can be improved by interpolating to finer and finerpixels. Noise in the data can be removed using a low pass filter or amedian filter, and the mapping of voltage level to brightness can beoptimized for motion estimation purposes. Typically, logarithmiccompression is used on the tracking arrays 20, 22, and this logarithmiccompression can be optimized independently of the logarithmiccompression used for the image data from the image data array 18. Theparticular mapping that is used can vary widely according to theoperator's wishes. It is likely that in many cases motion detection willfunction most efficiently using a different mapping than that used bythe operator for the image data. If desired, the system 10 can vary themapping used internally for the data from the tracking arrays 20,22until a mapping is found that provides high quality motion detection.

Three-Dimensional Volume Filling Computer 36

Many approaches can be taken in aligning the image data frames toprovide the desired three-dimensional reconstruction. One example isshown schematically in FIGS. 17-19. In this example, the image dataframes prior to reconstruction are shown schematically in FIG. 17. Theimage data frame for the central plane is inserted at a plane alignedwith the center of the volume, as shown in FIG. 18. Working outwardlyfrom this center plane, successive image data frames are inserted intotheir appropriate XYZ locations, as shown in FIG. 19. Once all frameshave been inserted, intermediate points are calculated usingthree-dimensional linear interpolation techniques relying on the eightclosest known data points, arranged as a cuboid around the point to beinterpolated. Such three-dimensional manipulation techniques are known,and are therefore not described in detail here,

One approach is to use the scan conversion interpolation methoddescribed by Leavitt in Hewlett Packard Journal, October, 1983, pp.30-34, adapted for use in three dimensions. The approach described byLeavitt operates with data in a two-dimensional plane. Data in threedimensions can be treated in two successive two-dimensional operations.Image plane data can be scan converted and interpolated as described byLeavitt, using pixel spacings that match the requirements of thethree-dimension scan conversion. Then an orthogonal two-dimensional scanconversion can be performed at each azimuthal position to fill out thevolume in the perpendicular direction. The Leavitt technique assumesthat the axis of rotation for successive two-dimensional images iscorrectly aligned. If this is not the case, other volumetricreconstruction methods can be used. Suitable reconstruction methods arewell-known and are used with conventional magnetic sensor-based systems.

If successive frames are detected to be too close together, some of theframe data may be discarded. If there are a large number of frames, itis possible to keep only those which happen to be spaced close to somedesired uniform spacing. For example, if the desired spacing forreconstruction is 2° and data is acquired at 0, 1.2, 2.1, 2.8, 3.9degrees, then frames 1, 3 and 5 can be approximately assumed to be thecorrect frames lying at approximately 0, 2, and 4 degrees. The errorfrom the approximation is insignificant and may result in simplicity inthe reconstruction.

Translation Detection Techniques

It should be noted that a component of motion within the image dataframe can also be detected using the techniques discussed above. Thiscomponent of motion will be parallel to the azimuthal axis, and can beadded to the motion detected with the tracking arrays, which isperpendicular to the azimuthal axis. Since the image data from the imagedata array 18 will move significantly due to motion of the elevationplane, it is preferred that motion detection be over-sampled in thisplane. Preferably, only a measured net motion is applied during thereconstruction, as discussed above in conjunction with FIG. 14.

Also, motion can be detected in the plane of the image data frame whenthe transducer 18 is moved in translation along the azimuthal axis, asopposed to the rotational sweeps discussed above. With this approach, anextended field of view can be provided by axially shifting thetransducer 16 along the azimuthal axis, without rotating the transducer16. If the only interest is in tracking such linear translation, thetracking arrays 20, 22 are not required.

As used herein, extended field of view denotes a system which storesimage data from a transducer array as the transducer array is shiftedaxially along its azimuthal axis. Data from the array at variouspositions along the azimuthal axis are then registered forreconstruction to form an extended field of view image.

The extended field of view discussed above can be reconstructed withdata from either one of the tracking arrays 20, 22 or the image dataarray 18. More logically, the image data array 18 is used, because theimage data array 18 is optimized for image quality. The image data arrayis then translated with respect to the target tissue, with its azimuthalaxis oriented parallel to the line of motion. Image motion detectionusing the techniques described above is performed on image data from theimage data array 18. Successive frames of image data from the array 18are stored, along with displacement information defining the motionbetween frames of image data.

Once the sweep of image frames has been completed, the displacementinformation is used, starting with the most recently acquired image dataframe, to register successive ones of the image data frames with respectto one another in proper alignment, in the tissue sense. The older imagedata is then superimposed on the newer image data. Typically, most ofthe older data will almost exactly match the newer data, but a smallnon-overlapping region will be present which represents data acquired atthe older image position which could not be acquired at the newer imageposition. Preferably, during the writing of the older data over thenewer data, only the non-overlapping region is written. This approacheliminates redundant image writing operations. This procedure is thencontinued for progressively earlier frames of image data until all ofthe frames in the sweep have been reassembled for display.

There is a potential that attempts may be made to write beyond thelimits of memory. If this situation is detected the image data can bescaled to make it smaller to fit into the available screen memory.Scaling may be achieved by remapping the pixels to a new memory usinggeometric transformation and interpolation, as is well known in thecomputer graphics field.

The following methods can be used either alone or in varioussubcombinations to enhance motion detection.

1. Frequency selection. In one example, every Nth frame is used formotion detection. On every Nth frame a different frequency is used toexcite the transducer elements of the image data array 18 in order tomodify the speckle pattern in such a way as to enhance the capability todetect motion. Since only relative motion is required, it may bepreferable to use a higher frequency to obtain higher resolution speckledata at every Nth frame. Higher frequencies are associated with reducedpenetration depths, but as discussed above this may not be a drawbacksince only shallow target data may be required for motion detection.Alternately, higher frequencies may make speckle highly positionsensitive. This may be undesirable if it causes the motion detectionoperation to fail due to an inability to track features that move in andout of the elevation plane of the image. In this case, it may bepreferable to use a lower frequency for the motion detection frames.

2. Bandwidth adjustment. As above, selected ones of the image dataframes may be optimized for motion detection, in this case by optimizingthe bandwidth as well as or instead of the center frequency for motiondetection.

3. Elevation focusing. When a 1.5 dimension array is used, elevationfocusing can be adjusted during the sweep to optimize some frames forimage data collection and other frames for motion detection. Forexample, to reduce the impact of elevational motion, it may bepreferable to defocus the beam slightly in those image data frames usedfor motion detection.

4. Azimuthal focusing. Similarly, it may be preferable to defocus theacoustic beam slightly in azimuth for selected frames to stabilizemotion detection without degrading motion detection excessively.

When any of the four approaches discussed above are used, the image dataframes that are optimized for motion detection need not be displayed onthe screen. Instead, only the image data frames that are optimized forimaging can be used.

Since during a slow sweep there is redundant data (multiple data pointsfor individual positions within the tissue), it may be preferable to mixthe use of different frequencies, bandwidths and focusing methods tocollect redundant motion detection data. After multiple redundant datais collected, it can be averaged to arrive at a motion estimate which ismore reliable and less vulnerable to error.

By way of example, if the array is moved at two millimeters per second,the required minimum motion which can be usefully resolved is 0.5millimeters, and the frame rate is forty frames per second, there willbe ten frames between times when useful motion detection can beobtained. Also, we need only one image data frame for each of these tenmotion detection frames. If we consider the first twenty frames, frames1 and 11 can be image data collected using user-determined centerfrequency and bandwidths. Frames 3 and 13 can be motion detection frameswith first alternative center frequency and bandwidth. Frames 5 and 15can be motion detection frames with second alternative center frequencyand bandwidth, and frames 7 and 17 can be motion detection frames withthe first alternative center frequency and the second alternativebandwidth. Frames 9 and 19 can be motion detection frames using achanged focusing scheme. Frames 2, 4, 6, 8, 10, 12, 14, 16, 18 and 20are captured as if they were image frames and are displayed, but are notstored or used for tracking. The detected motion from frames 1, 3, 5, 7and 9 with respect to frames 11, 13, 15, 17 and 19, respectively, maythen be averaged or sorted so as to eliminate motion estimates which aresignificantly different from other estimates.

All of the techniques described above in connection with extended fieldsof view can be adapted for use with motion detection in angular sweepsas described above.

In the case of a complex motion involving motion in both the planes ofthe tracking arrays and motion in the plane of the image data array, theactual motion may be calculated approximately from the sum of thetracking array motions and the image data array motions, separatelymeasured. If after the sum of such motions is obtained it is found thatthe second motion to be applied has caused the first motion to have beenmodified, a recursive operation can be employed until athree-dimensional motion is found which correctly satisfies the detectedmotions in all arrays. A Monte-Carlo method for finding the componentssatisfying a complex motion may be used. Additionally, new motions maybe estimated from the last frame-to-frame motion which was calculated.As a last resort, the system can find that it is unable to determine aset of motion component vectors which satisfy the detected motions andcan signal the operator to repeat the last sweep. The simpler, smoother,and purer that image movement is in any given image plane, the easier itis to implement motion detection.

Alternative Embodiments

A simpler device for implementing the bi-plane imaging techniquesdiscussed above can be constructed as follows. This device also uses alinear array for the image data acquisition and two small linear arraysof equal pitch that are mounted to move with the larger linear array.The two small linear tracking arrays are coupled to a connector similarto that used by the image data array, but the transducer elements aredivided so that the transducer elements for each tracking array arecoupled to the transducer connector near the opposite ends of theconnector. For example, for a 128 conductor connector and a 10 elementtracking array, the conductors 10-19 and 100-109 can be used for the twotracking arrays, Both the image data array and the combined trackingarray are provided with a connector of the same type and are designed tooperate with the same beamformer. The two connectors are inserted intothe left and right ports of a suitable ultrasonic imaging machine, suchas the Acuson Model XP, and an image storage device such as a VCR isprovided. As the array is rotated slowly the operator presses the arrayselect switch (left versus right) repeatedly at a high rate and theresulting images are stored for off-line analysis. At the end of thesweep, the VCR tape is played back on a frame-by-frame basis. Successivesets of frames, each set consisting of a frame from the image data planeand each tracking image plane, are transferred via standard videotransfer techniques to a computer memory. The computer is programmed toanalyze the tracking image plane data for motion detection using themethods discussed above. The tracking image will in fact appear as twoimages, one on each side corresponding to the two spatially separatedtracking arrays. The two images are analyzed separately.

As a preferred modification to the simplified system describedimmediately above, the ultrasonic imaging machine is preferablyprogrammed to perform the array switching operation and to transfer oneset of frames from each sequential sweep to computer memory or to asuitable storage device.

Alternative Transducers

Many variations are possible on the preferred transducers describedabove. For example, crossed arrays of the type described in HashimotoU.S. Pat. No. 5,327,895 and Schaulov, et al., Ultrasonics Symposium, pp.635-638 (IEEE, 1988) can be adapted for use with this invention. Suchcrossed arrays typically include a PZT plate which is diced inperpendicular directions. The kerfs are filled with a low-durometerpolymer such as a low-durometer epoxy. Linear electrodes (defining thearray elements) are applied to the top and bottom surfaces, and theelectrodes on the top surface are oriented perpendicularly to theelectrodes on the bottom surface. When using the array in one direction,all electrodes on the top surface are grounded and the electrodes on thebottom surface are excited with the phased excitation pulses. Theelectrodes on the bottom surface are then monitored for detected echoes.When using the array for the other direction, the bottom surfaceelectrodes are grounded and the top surface electrodes are operated in aphased manner.

Both arrays in a crossed array configuration may be used simultaneously.For example, one array may be operated using a different centerultrasonic frequency than the other. Also, both arrays may be usedindependently of one another, as indicated by the principle of voltagesuperposition, thereby accommodating simultaneous operation of botharrays.

FIGS. 23-26 show schematic views of four crossed arrays that can be usedto collect both image data and tracking information. In the transducer100 of FIG. 23 the elements 102 form a conventional one-dimensionalarray that can be used to collect image data. Crossed elements 104 atboth ends of the transducer 100 can be operated as tracking arrays asdescribed above. The transducer 100′ is similar to the transducer 100except that the crossed elements 104′ are provided only at one end.

The crossed array 100″ of FIG. 25 is also similar to the array 100 ofFIG. 23, except that the crossed elements 104″ are provided only at thecenter of the transducer. FIG. 26 shows a transducer 100′″ which issimilar to the transducer 100″ of FIG. 25, except that the crossedelements 104′″ extend over the entire length of the array.

As shown in FIGS. 23-26, one or more tracking arrays can be integratedwith an image data array using the crossed array technique. The crossedarray is operated in the normal manner along the long, azimuthal axis toobtain 2-D image data. The crossed arrays are operated to obtain dataalong the elevation direction to obtain the perpendicular tracking data.In this way the footprint of the transducer is minimized. If desired,the tracking data may be acquired from the same volume of tissue as thatbeing interrogated to obtain the image data.

FIG. 27 provides a schematic view of another transducer 110 whichincludes an image data array 112 and a single tracking array 114. Inthis case the tracking array 114 is oriented perpendicularly to theimage data array 112. As shown in FIG. 27 the tracking array 114 islaterally offset from and centered with respect to the image data array112.

FIGS. 27a through 27 e show five alternative transducers, each includinga single image data array 112 and at least one tracking array 114. Inthe transducer of FIG. 27a there are two tracking arrays 114, bothlaterally offset from and centered with respect to the image data array112. In the transducer of FIG. 27b there is a single tracking array 114that is co-linear with and axially spaced from the image data array 112.

The transducer of FIG. 27c includes two tracking arrays 114, bothpositioned to the same side of the image data array 112 near oppositeends of the image data array 112. The transducer of FIG. 27d is similarto that of FIG. 27c, except that the two tracking arrays 114 arepositioned on opposite sides of the image data array 112. The transducerof FIG. 27e includes four tracking arrays 114, two on each side of theimage data array 112, near respective ends of the image data array 112.

FIGS. 27f through 27 j show additional transducers that can be used withthis invention, each including a single image data array 112 and atleast one tracking array. In the transducer of FIG. 27f, each trackingarray includes two single-element transducers 114′. Each of the trackingarrays is situated near a respective end of the image data array 112,and the individual single-element transducers 114′ are elongatedelements oriented generally parallel to the azimuthal axis of the imagedata array 112. The two single-element transducers 114′ in each trackingarray can be operated as an interferometric, two-element transducerarray. The single-element transducers 114′ can be fired using anarrow-band transmit signal, for example either in phase or 180° out ofphase. Receive signals generated by the two single-element transducerelements 114′ can then be added together to create a summation signal.Consecutive summation signals can then be compared to one another withvarious offset delays. The offset delay which gives the best correlation(in a least squares sense for example) is a function of the motion ofthe transducer between the respective firings of the transducers 114′.Using this best correlation time delay, and taking into account thesound propagation velocity through the two way path, one can thendetermine the associated physical displacement of the transducer betweenthe associated firings, as described in detail above.

The comparison operation may be performed in either the time or thefrequency domain. If the frequency domain is used, complex values orphase values are preferably compared. For many applications a timedomain comparison will be simpler. Because the two single-elementtransducer elements 114′ within a single tracking array are relativelywidely spaced, they provide the benefit of relatively good lateralresolution. High grating lobes will typically be associated with thisconfiguration, but it is envigaged that high side lobe levels will beless objectionable for use in an automated tracking algorithm than foruse in a gray scale display intended for human interpretation.

In some applications the method described above would principally detectmotion parallel to the beam axis. In these cases it may be preferable toform the beams with a substantial component in the direction of theexpected motion. Since motion is principally in the lateral direction(transverse to the azimuthal axis of the image data array 112), oneoption is to form the beams at +/−45° or +/−60° from the central imageplane of the image data array 112.

As shown in FIG. 27g, the single-element transducer elements 114′ may besuperimposed on the image data array 112. This can be done using crossedarrays as described above, or by placing one of the arrays physicallyover the other. In this case the upper array should be as nearlytransparent as possible with respect to ultrasonic radiation emitted bythe lower array. In the example of FIG. 27g the image data array 112 maybe an elongated, one-dimensional, ceramic-based array, and thesingle-element transducer elements 114′ may comprise PVDF arrays mountedwith perpendicularly-oriented elements on top of the image data array.In fact, the PVDF tracking array may perform the function of a matchinglayer for the image data array. A non-conducting plastic film may beplaced between the superimposed arrays to prevent cross talk orshorting.

As shown in FIGS. 27h and 27 i, the single-element transducer elements114″ may take the form of circular elements which may be preferablyfocused at some range, as for example 40 millimeters. In this case, thereceive signal generated by a single one of the single-elementtransducer elements 114″ is compared with subsequent firings of the samesingle-element transducer element, and beamforming operations areeliminated. In this case, each individual one of the single-elementtransducer elements 114″ acts as a respective tracking array, and theterm “tracking array” is intended broadly to encompass suchsingle-element arrays.

As shown in FIG. 27j, each individual tracking array 114′″ may take theform of an annular array. Annular arrays are well known for use inmechanically scanned transducers, and they provide the advantage thatthey can be focused in transmit and dynamically focused in receive toproduce a high quality beam profile. Each annular array 114′″ istypically composed of cross-diced PZT ceramic with an epoxy filler.Chromium/gold spatter electrodes are applied to the PZT ceramic, andthese electrodes are etched selectively (or mechanically scribed) toisolate approximately 8 discrete electrodes on one side. The reverseside has a continuous metalized layer and is grounded. As describedabove, the tracking arrays 114′″ may be oriented at ±45° or ±60° forexample with respect to the central plane of the image data array 112. Alow loss, non-refractive filler polyurethane such as the resindistributed by Ciba-Geigy as resin no. RP6400 or other suitablediphenylmethane diisocyanate polyurethane can be used to fill the spacebetween the transducer elements 114′″and patient tissue.

Transducer geometries that place the tracking arrays alongside the imagedata array may reduce the overall length of the transducer, which mayprovide advantages in some applications.

As shown in FIGS. 28 and 29, a transducer 120, 120′ suitable for use informing an extended field of view as described above can utilize animage data array 122 and a tracking array 124, wherein the transducerelements of the two arrays 122, 124 are parallel to one another. In thetransducer 120 the tracking array 124 is laterally offset from andcentered with respect to the image data array 122. As shown in FIG. 29,in the transducer 120′ the image data array 122′ and the tracking array124′ are collinear and separated from one another. With either of thetransducers 120, 120′ frame-to-frame motion can be determined using thetracking arrays 124, 124′ and the compound image can be assembled fromimage data from the image data array 122, 122′.

In any of the foregoing embodiments, some or all of the tracking arraysand the image data arrays may be formed as non-planar linear arrays suchas curved linear arrays for example. Separate, oriented linear arraysmay be used for each acoustic line if desired.

Alternative Optimization Techniques For Tracking Data

FIG. 30 is a block diagram of an alternative ultrasonic imaging system10′ that is in many ways similar to the system 10 of FIG. 1. The samereference numerals have been used for comparable elements in the twodrawings, and the following discussion will focus on the differences.The system 10′ stores image data from the image data array 18 or one ofthe tracking arrays 20, 22 in a raw image data buffer 15. The datastored in the buffer 15 can be I, Q data or IF data, for example. Ofcourse, the term “image information” is intended to refer broadly toinformation that varies with spatial variations in the target, and inmany cases image information is never displayed as an image. Preferably,the raw image data stored in the buffer 15 includes tracking data storedwith wide band ultrasound pulse operation. Image data from the buffer 15is passed to a detector 21 through one of four alternative blocks. Block17 is an all-pass block which performs no filtering, and which istypically used for image data from the image data array 18. Trackingdata from one of the tracking arrays 20, 22 can be passed through one ofthree filters 19, 19′, 19″, which can have varying filtercharacteristics. For example, filter 19 may be a low-pass filter andfilter 19′ may be a high-pass filter. Filter 19″ may be a band-passfilter centered in any desired portion of the frequency spectrum.

The detector 21 can be any conventional detector, and it supplies adetected output signal to the scan converter 24. If desired, trackingdata from the tracking arrays 20, 22 can be processed multiple times,using different ones of the filters 19, 19′, 19″. Of course, more orfewer filters can be used in alternative embodiments.

The scan converter 24 preferably uses the well-known histogramequalization method to maximize contrast in the tracking image data.Histogram equalization is discussed for example in Gonzales and Woods,Digital Image Processing, Addison-Wesley, 1993, pp. 173-178, as well asin Kim U.S. Pat. No. 5,492,125. The scan converter 24 may use a 2Dlow-pass filter and/or a 2D high-pass filter to smooth or otherwiseprocess the tracking image data.

Image frames from the scan converter 24 are stored in the frame buffers30, 32, 34, and tracking sets of image data from the buffers 32, 34 areselectively applied to the motion estimator 38. The motion estimator 38estimates the relative motion between selected tracking image sets andapplies these estimates of motion to a motion smoothing/summing block40′.

The motion smoothing/summing block 40′ processes motion estimates frommultiple motion detection operations. Preferably, each motion detectionestimate is associated with a quality factor indicative of theconfidence level associated with the motion estimate, For example, asuitable quality factor may be value of the minimum sum of absolutedifferences associated with a particular motion estimate. The block 40′compounds multiple motion estimates using a weighted sum. For example,consider the situation where three motion estimates are available withthe following values and associated quality factors Q:

Estimate 1 5 Pixels to the right Q = 0.9 Estimate 2 3 Pixels to theright Q = 0.8 Estimate 3 2 Pixels to the left Q = 0.2

In this example, a high value of Q is associated with a high confidencelevel. The block 40′ can form a weighted sum of these three motionestimates as follows: $\begin{matrix}{{{Motion}\quad {Estimate}} = \frac{\left( {5 \times 0.9} \right) + \left( {3 \times 0.8} \right) + \left( {{- 2} \times 2.0} \right)}{\left( {0.9 + 0.8 + 2.0} \right)}} \\{= {3.4\quad {Pixels}\quad {to}\quad {the}\quad {{right}.}}}\end{matrix}$

The previous motion store 41FIG. 3 is used to store previous motionestimates until they are needed in forming the weighted sum describedabove. Other methods for using multiple motion estimates are describedbelow.

Other techniques that can be used to improve the quality of the motionestimate include the use of multiple transmit zones to acquire the bestquality tracking image data and the use of a frequency-dependent focusas described in Hossack U.S. Pat. No. 5,608,690, assigned to theassignee of the present invention, to acquire the best quality trackingimage data.

As pointed out above, the acquisition of image data via the array 18 canbe time multiplexed with the acquisition of tracking data via thetracking arrays 20, 22. In one alternative embodiment the timeseparation between consecutive tracking data frames is controlledadaptively to provide motion estimates that fall within a desired range.As explained above, if the motion estimate (i.e. displacement) betweenconsecutive tracking frames is excessive, there is a danger that thedisplacement may exceed the measuring capabilities of the motionestimator 38. Conversely, if the motion estimate between consecutivetracking frames is too low, excessive computational time may be used increating motion estimates. Additionally, if the motion estimates aresmall, the relative errors will be large, and compound errors may reachundesirable proportions. In order the avoid these problems, a controllercan be provided as described above to determine the number of image dataframes that are collected between consecutive tracking frames, and thiscan be accomplished in an adaptive manner as shown in FIG. 31.

In FIG. 31 the variable N is used to specify the number of image dataframes that are collected between consecutive tracking frames. As shownFIG. 31, N is initially set to a constant K1, and the controller thenwaits for a new motion estimate from the motion estimator 38 of FIG. 3.If this motion estimate is above a desired range, than N is reduced bythe amount Δ. Conversely, if the motion estimate is below the desiredrange than N is increased by Δ. Once N has been revised if necessary,the controller then waits for a new motion estimate from the motionestimator 38. In this way the motion estimate is maintained within adesired range automatically, and problems associated with excessivelylarge motion estimates or excessively small motion estimates areavoided.

Image Transfer Optimization Techniques

The system 10′ of FIG. 30 is well-suited for use in situations where aremote computer is used to perform motion estimation, either in realtime or after a delay. In this context remote may mean that the motionestimating computer is connected via a cable or other data link. Asshown in FIG. 30 image data frames from the buffer 30 can be compressedusing any suitable compression technique such as JPEG prior to transferto the potentially remote site. After the image data has been receivedat the potentially remote site, it is decompressed as shown in block 35.Similar compression and decompression blocks can be interposed betweenthe buffers 32, 34 and the motion estimator 38. For example, remotemotion estimation and 3D volume reconstruction can be performed on aremote workstation such as the AEGIS workstation of Acuson Corporation,the assignee of the present invention.

Alternative Motion Estimation Techniques

In the event that motion between consecutive tracking frames is small(less than one pixel) there is a danger that motion may be mis-estimatedif each tracking frame is compared with the immediately precedingtracking frame. For example, if the separation between adjacent trackingframes were ⅓ of a pixel, the comparison of each tracking frame with theimmediately preceding tracking frame would detect no motion. In order toovercome this problem, the motion estimator 32 of FIG. 30 preferably iscontrolled using the algorithm of FIG. 32. In this algorithm, N definesthe frame number of the image region used as a reference in estimatingmotion, and the symbol N+i is used to designate the frame number of theimage region that is to be compared with the reference image region. Asshown in FIG. 32 the first step is to set i=1 and then to determinewhether motion was detected between image region N and image region N+i.If not, i is incremented and control is returned to block 130. Oncemotion is detected in block 130, the reference image frame is updated toequal N+i, i is reset to 1, and control is returned to block 130.

For example, consider a sequence of tracking frames 1, 2, 3 and 4.Assuming no motion is detected between frames 1 and 2, then frame 3 willbe compared with frame 1. Assuming no motion is detected between frame 3and frame 1, then frame 4 will also be compared with frame 1. Thisprocess is repeated until motion is detected, and it is only then thatthe reference frame is updated to the new frame at which motion wasdetected. In this way, multiple subpixel motions do not go undetected,because subpixel motions eventually sum to the point where they becomedetectable.

The selection of the reference image region does not necessarilycorrespond to an entire frame. If frame motion is zero at the top andnon-zero at the bottom of the frame (as might be the case with afan-like sweep), then the older frame portion is preferably kept as thereference for motion detection at the top of the frame, while the bottomof the frame is updated once motion is detected in that region.

Since the tracking arrays 20, 22 are principally swept along thesurface, the majority of motion will be in the elevation directionrather than the depth direction. During the search for the minimum sumof absolute differences (MSAD), it is preferable to make the searchregion rectangular rather than square. For example, instead of searchingan area 64×64 pixels, a 128×32 pixel area can be searched in the sametime (128 lateral search pixels and 32 depth search pixels). Similarly,if the maximum motion is 64 pixels, then by limiting the area of thesearch to 32×64 pixels, the search time is greatly reduced.

Additionally, frame motion can be interpolated to further reduceprocessing time. For example, if tracking frames 1, 3 and 5 are used formotion detection (rather than 1, 2, 3, 4 and 5), then detected motionbetween frames 1 and 3 can be interpolated to determine the motion forframe 2. In this case the interpolated motion for frame 2 would beone-half of the detected motion between frames 1 and 3. This methodallows reduced processing time to be spent for motion detection, butallows all acquired image data to be employed in the highest quality 3Dimage. Interpolation of this type may be performed in the 3D volumetranslation/interpolation block 36 of FIG. 30.

Various alternative approaches can be used in the motion estimator 38 tofurther reduce processing time. For example, if a small set of acousticlines are transmitted, received and stored as one-dimensional RF orbaseband signals, then vector components of motion along each of theselines can be estimated by correlating successive sample sequences alongthe lines. In this way the vector component of motion in each of theline directions can be determined, and these vector components of motioncan be summed to create the final two-dimensional motion estimate, Ifdesired more than two acoustic lines may be used in the set, but thefollowing example uses two perpendicular acoustic lines.

For example, as shown in FIG. 33, the tracking array 20 can be used tostore two acoustic receive lines 140, 142 which are perpendicular toeach other. These lines can be considered to make up a single frame oftracking data. The line 140 can be cross-correlated between two separateframes of tracking data to find the vector component of motion along thedirection of the arrow 144, and similarly the lines 142 in these twoframes of tracking data can be used to determine the vector component ofmotion in the direction of the arrow 146. These two components of motioncan then be summed as vectors to estimate the two dimensional motionbetween the two frames of tracking data. Cross-correlation techniquessuitable for adaptation to the current task are described in EngelerU.S. Pat. No. 4,937,775, O'Donnell U.S. Pat. No. 4,989,143 and Wright,et al. U.S. Pat. No. 5,570,691. The Wright, et al. patent is assigned tothe assignee of the present invention.

Preferably, a register is used to store the complex sampled beam datafrom one firing in one of the two directions. When, an instant of timelater, the same beam is formed again, the resulting sampled beam data iscross-correlated with the data in the register. From the position of thecross-correlation peak, the relative time delay between the two signalsis determined. The cross-correlation process may operate on one or moreportions of the available line data. By detecting motion viacross-correlation at a number of points along each line, separatemotions can be determined at the top and bottom of the image, and hencerotations as well as translations can be estimated. From the time delaydetermined by the cross-correlation peak, the component of transducermotion (parallel to the beam axis) required to cause the measured timedelay is inferred from the known speed of sound in tissue and takingaccount of the fact that the delay is related to two-way path length.This process is repeated for the other line (preferably orientedperpendicularly to the first line) to find the other vector component ofmotion. These two vector components are then summed to find theestimated actual transducer motion at that point along the array.Similarly, the process is typically repeated at the second side of thearray for the second tracking array.

In one embodiment, the two lines (and therefore the two beams) areoriented at ±45 degrees, and there is therefore a strict requirement onelement spacing if grating lobes are to be avoided. In anotherembodiment the two lines are oriented at ±60 degrees to enhance theaccuracy of motion detection along the skin of a patient. Preferably,the transducer elements should be spaced at one-half wavelength or less.This requirement may encourage the use of a lower frequency for thetracking arrays than for the image data array. However, it is possiblethat the cross-correlation technique will be able to track delay to asmall fraction of a wavelength. As before, the comparison between oneset of tracking data and the next may not be made on every acquired setof tracking data, but rather on a time-spaced subset. For example, ifmotion is fast, motion estimates can be made between every consecutivepair of sets of tracking data, but if motion is slow a longer timebetween motion estimates is allowed.

Fast Motion Detection Search Techniques

A variety of methods are available for speeding up the search for theestimated motions. These methods can be used with any of the embodimentsdescribed above.

1. Either the system controller or a user input may vary the block sizeto be searched. Large block sizes require more computation, but forcertain image types (e.g. noisy data) large blocks will give a higherquality result (i.e. lower minimum SAD versus mean SAD).

2. The system adaptively changes the search area based on the previousmotion estimate. If the last motion was 10 pixels to the right then thesearch may be over an area from 0 pixels to the right to 20 pixels tothe right. i.e. the search area is centered on the expected motion asindicated by the previous motion measurement).

3. If the detected motions exhibit little variation (e.g. 7, 8, 10, 9pixels to the right in successive motion measurements, as opposed to 5,15, 8, 12 pixels) then the system may use a smaller search area sincethere is a relatively high degree of confidence that the motion will bebetween 7 and 10. Conversely, if successive motion estimates are widelyvarying, then the system preferably uses a larger search area in orderto maximize assurance of a successful search (i.e. the motion is notbeyond the limits of the search area). In the above example, if pixelmotions are 7, 8, 10, 9 then we use a 15×15 search area but if pixelmotions are 5, 15, 8, 12 then we use a 30×30 search area.

4. Since most motion is expected in the lateral direction rather than inthe range direction (because the transducer is drawn across the skinsurface) the search area may be asymmetric. e.g. +/−5 pixels in rangedirection and +/−20 pixels in the lateral direction.

5. The search may be performed at multiple levels of spatial resolution.Initially, the search is made coarsely along the lateral direction only,e.g. test every second pixel location. Once the approximate lateraloffset has been thus detected, a fine search (every pixel location) ismade in both the lateral and range direction,

6. Hierarchical motion detection can be used based on multiple levels ofsignal level resolution. Initially, only the most significant 2 or 4bits associated with each pixel intensity level are used to find theposition with the minimum SAD. Once the location has been approximatelyfound, a second level of search is performed in that region using allthe bits, typically 8. More than two levels of hierarchy can be used.

Higher Resolution Tracking Techniques

The finest level of motion detection may be enhanced by interpolatingadditional pixels between the available pixels. An alternative methoddescribed by Li and Gonzales, IEEE Trans on Circuits and Systems forVideo Techn., 6, 1, pg. 118 (February, 1996), calculates the motionestimate to sub-pixel resolution based on the values of the neighboringSAD values to the one with the minimum SAD.

Techniques for Combining Motion Estimates

The final estimate of transducer motion is preferably based on acomposite of multiple inputs. Preferably these inputs are weighted sothat those that appear to possess the greatest quality (or certainty)are given the most weight. Inputs which are contradictory with themajority of inputs are either eliminated from the composite calculationor are given very small weightings.

Firstly the ratio of minimum sum of absolute differences (“min_SAD”) tomean sum of absolute differences (“mean_SAD”) is used as a qualityfactor; A low ratio indicates a high quality result and a result closeto 1.0 indicates an unreliable estimate. In practice a good ratio israrely less than 0.3. Assuming that ratios lie in the range 0.3 to 1.0,we can convert this into a weighting function in the range 0.0 to 1.0,where 1.0 means an ideal (high certainty) result and 0.0 means anunusable result:

Weighting_MSAD=(1−(min_SAD/mean_SAD))/0.7.

If the minimum observable SAD is <0.3, then this equation can bemodified:

Weighting_MSAD=(1−(min_SAD/mean_SAD))/(1−min_observable_SAD).

A second quality factor is based on the similarity of the present motionestimate to the previous estimate. This approach is based on theobservation that during a smooth scan typical of that which would beused in practice, the actual relative motion between one set of trackingdata and a subsequent set of tracking data is similar. If a motionestimate predicts a reversal in motion, then it is probable that it is abad estimate. Causes for bad estimates may include a noisy image, poorpixel contrast, or the presence of large amounts of flowing blood.Notice that the previous motion estimate that is used as a reference maybe either the raw estimate of motion from the MSAD operation or theprevious smoothed and weighted estimate. Preferably, the smoothedprevious motion estimate is used as the reference to which the mostrecent raw motion estimate is compared.

In the current example the degree of similarity between two estimates ofmotion is calculated as follows. This example is for Y (elevation)direction motion, but it can be applied to Z (depth) direction motion.Weighting_seq is the weighting factor that is generated based on acomparison of sequential estimates.

Weighting_seq=1−[abs (Ycurr−Ylast)/(abs (Ycur)+abs(Ylast))],

where Ycurr=the current estimate of Y motion, and

Ylast=the last estimate of Y motion, smoothed.

For the initial motion calculation, no measure of similarity to aprevious estimate is possible, so the initial motion-related weightingmust be given an arbitrary value such as 0.5.

A composite weighting factor (“Weighting_comp”) is then formed.Depending on experience with real tissue scanning, one may select tobias the weighting more to either the MSAD quality or to the sequentialquality. Currently 0.75 of the total weighting is related to the MSADquality and 0.25 to the sequential quality.

Weighting_comp=0.75×Weighing_MSAD+0.25×Weighting_seq.

These weightings are calculated for all the points for which motionestimates were made. In an example 6 motion estimates were made in therange direction. Each used a 48×48 pixel block.

Since for a real motion the equation describing the motion as a functionof depth must follow a straight line, we can fit the obtained motionestimates to a straight line. Preferably, the motions are fitted using aweighted least squares method wherein the weightings are those describedabove for each of the range points for which a motion estimate isavailable. This process is repeated for both Y direction motion and Zdirection motion.

Finally, having determined the fitted motions, we have the line definedin terms of an intercept (c) and slope (m): Y_motion=mZ+c.

Similarly, the Z motions as a function of the Z observation points arecalculated:

Z_motion=mZ+c (different m and c).

Although one would typically expect Z_motion to be a constant as afunction of Z, it is not in the case of transducer rotation about theazimuthal axis.

This fitted value may be further smoothed based on the average of theparameter Weighting_comp. Hence, in the presence of a very poor qualityestimate, the current estimate is based principally on the previous(rather than the current) estimate:

m_mod=fact*(mean (Weighting_comp))*m+(1-fact*(mean(Weighting_comp))*m_last

c_mod=fact*(mean (Weighting_comp))*c+(1-fact*(mean(Weighting_comp))*c_last

c the current estimate of intercept

c_last the intercept from the previous pair-of-frames motion estimatec_mod the modified intercept m the current estimate of slope m_last theslope from the previous pair-of-frames motion estimate m_mod themodified slope fact a factor to determine how much significance toattach to the weighting (fact = 1.0 presently)

Alternatively, the final Weighting_comp may be determined using fuzzylogic. A fuzzy logic control block takes as inputs Weighting_MSAD andWeighting_seq and combines them to form an output which isWeighting_comp. Inputs Weighting_MSAD and Weighting_seq are firstassigned to classes (separate classes for Weighting_MSAD andWeighting_seq). These classes are ‘low’, ‘medium’ and ‘high’. Membershipis based on whether the input comes within each of three triangular-likeregions as shown in FIG. 34. The derivation of positions of the linesdefining in which classes particular measured values will reside isbased on experimentation. Although the illustrated regions of thedrawings are shown as triangular in shape, it should be noted that theregions may be shaped to follow any continuous function that isdetermined experimentally to give good results. The horizontal axis ofthe class function corresponds to the input value (Weighting_MSAD orWeighting_seq) and the vertical axis defines the degree of membership ofthe class. In this case, the same class membership diagram is used forboth Weighting_MSAD and Weighting_seq. A similar class of membershipdiagram is derived for the fuzzy output Weighting_comp, as shown in FIG.35. In this case the diagram has five regions—very low, low, medium,high and very high.

The following fuzzy rules can be applied to determine Weighting_comp; inthese rules a logical AND is assumed between Weighting_MSAD andWeighting_seq:

Rule No. Weighting_MSAD Weighting_seq Weighting_comp 1 low low very low2 low medium low 3 low high medium 4 medium low very low 5 medium mediummedium 6 medium high high 7 high low low 8 high medium high 9 high highvery high

Fuzzy rules are applied to determine the truth of the rules. Forexample, assume that Weighting_MSAD and Weighting_seq are 0.35 and 0.9respectively.

The 0.35 input results in 0.5 degree of class membership in ‘Low’ and0.5 degree of class membership in ‘Medium’. The 0.9 input results in 1.0degree of class membership in ‘High’.

Therefore Rules 3 and 6 are true but provide different values for theoutput Weighting_comp (‘medium’ and ‘high’ respectively). The outputsthat are possible for these rules are shown in FIG. 36.

Referring firstly to Rule 3, the low value of Weighting_MSAD is combinedwith a logical AND with the high value of Weighting_seq and the minimumvalue of the two expressions is taken as the truth level of Rule 3. The0.5 degree of membership of ‘low’ for Weighting_MSAD is less than the1.0 degree of membership of class ‘high’ for Weighting_seq. Hence thetruth level of the Rule 3 is 0.5.

Referring to Rule 6, the medium value of Weighting_MSAD is combined witha logical AND with the high value of Weighting_seq and the minimum valueof the two expressions is taken as the truth level of Rule 6. The 0.5degree of membership of ‘medium’ for Weighting_MSAD is less than the 1.0degree of membership of class ‘high’ for Weighting_seq. Hence the truthlevel of the Rule 6 is 0.5.

The ‘medium’ and ‘high’ labels for the Weighting_comp functionmembership in FIG. 36 are truncated at the truth levels defined by thefuzzy rules above. This is shown in FIG. 37.

A numerical output for Weighting_comp is derived using a centroiddefuzzification technique. An estimate of the center of gravity of theentire shaded region in FIG. 37 is made. In this case, the center ofgravity is at 0.625, and hence the output Weighting_comp is assignedthis value.

Having determined Weighting_comp, one can use the method shown above todetermine the weighted least squares fit and to determine to what extentthe current motion should be based on the current motion estimate or onthe previous motion estimate.

Of course, it will be apparent from the foregoing discussion that thedetected motions in the X, Y and/or Z directions are local motions, i.e.motions with respect to the current position and orientation of thetransducer and its arrays. Initially, at the start of the scan, it istypically assumed at the local X, Y and Z directions correspond to anassumed global axis system which remains constant throughout the motionof the transducer. If the transducer is rotated about the azimuthal axisof the image plane, as it might during a fan-like sweep, then the localZ motion (depth direction of the transducer) will rotate until itcontains a significant component in the global Y or elevation direction.With every detected motion of the transducer, the new position andorientation of the transducer in the global axis system are calculated.The orientations of the local X, Y and Z directions (i.e. azimuth,elevation, and range or depth of the transducer) with respect to theglobal axis system are updated. Therefore, subsequent analysis of motionin the local Z direction of the transducer is decomposed into componentsin the global Z and Y directions, for example.

By way of example, consider the situation where the depth or Z directionof the transducer has been rotated from initially pointing down inalignment with the global Z direction to being angled at 45° withrespect to the global Z direction. A motion in the local Z direction ofthe transducer of ten pixels is now decomposed into 10 cos (45°) in theglobal Z direction plus 10 cos (45°) in the global Y direction. In thisexample, the local Z direction is still orientated perpendicularly withrespect to the global X direction, and hence a local Z motion has nocomponent in the global X direction.

In general, the relation of the local axis directions with respect tothe global axis directions can be calculated using cosines, which arecontinuously updated as the transducer is swept through the volume.Preferably, these direction cosines are maintained continuously for allthree axes.

As used herein, the term “component of motion” is intended broadly toencompass translational components, rotational components, andcombinations thereof,

Data Overlapping Techniques

When two regions of image data are combined to obtain an extended fieldof view as described above, a smooth interpolation scheme is preferablyused at the boundaries of the data from different image data frames. Asshown in FIG. 38, variable weighting factors can be used to provide suchsmooth interpolation schemes. In FIG. 38, the right-hand edge of theprevious frame ends with pixel number 3 and the new portion of the framebegins with pixel number 4. The weighting factor used for the previousframe 1 varies from one within the old frame at pixels 0 and 1, smoothlydown to 0 within the new portion of the frame at pixels 5, 6 and 7.Conversely, the weighting factor for the subsequent frame 2 varies from0 within the old frame at pixels 0 and 1 gradually up to 1 for pixels 5,6 and 7 of the new portion of the frame. In general, compounding can beused for accumulated image data in any of the embodiments describedabove to reduce noise in the image. Similarly, compounded data may beused with tracking data to reduce noise.

Techniques for the Detection of Incorrect Estimates of the Component ofMotion

There are a variety of ways of using the motion estimates along the beamlines emanating from the tracking arrays to create a definition of theplane of the image array and the 3-D pixel locations for pixels on theimage array plane.

One approach is to fit straight lines using a least squares or weightedleast squares technique in which motion quality is used as a weightingfactor. Based on these lines a number of points can be identified alongwith their associated 3-D motions. For example, four points can beselected at the top left, bottom left, bottom right and top rightportions of the region, and bi-linear interpolation can be used to findpixel locations. From the motion estimates along the lines the equationfor a plane (Y=a+bX+cZ) can be fitted. The quality of the fit may bemeasured by a least squares or weighted least squares technique. Thenusing the four key points identified previously, find the equation for aplane, i.e. fit three equations Y=a+bX+cZ for three points to find a, b,c using for example matrix inversion techniques. This step is repeatedfor a different set of three of the four key points. If the ratios a1/a2(where a1 and a2 are the ‘a’ values derived from different sets of threepoints), b1/b2, or c1/c2 exceed a threshold (either too large or toosmall), the system can be programmed to warn the user via a video outputthat the plane is skewed, and that a rescan is appropriate.

The sum of squared errors, in either or both the line and plane case,may be used as a measure of low quality motion estimates and used eitherinternally to adapt the motion estimate (i.e. use larger block sizes ormake more motion estimates) or supplied to the user as a video oraudible warning, as for example prompting a re-scan. Similarly, it ispossible to accumulate the detected motion errors between successivemotion estimates and to compare the cumulative motion error against apreset threshold. When the cumulative motion error crosses thisthreshold, the system is preferably programmed to warn the user thatcumulative position error is unsatisfactory, and that a rescan ispreferred. Optionally, the cumulative motion error (which may be derivedfrom the sum of squared errors but is not necessarily exactly equal tothe sum of squared errors) may be displayed for the user.

Alternative 3D Viewing Techniques

As described above, multiple 2D slices can be reconstructed into a solid3D volume set. This is, of course, not the only approach that can beused to form the output display 46. For example, a simpler display maycomprise a displayed reference bounding box which is 3D in form althoughdisplayed as a 2D projection. Using the motion detection informationdescribed above, the position and angular orientation of all of theplanes of image data are calculated as before. However, in thisalternative a selected 2D slice is displayed oriented within the 3Dbounding box displayed on the screen. This can be done using the processdescribed in Keller, U.S. Pat. No. 5,353,354. Techniques for projectingimage data defined in 3 dimensions onto a 2-day screen are well known,as described for example in chapters 5 and 6 of Computer Graphics(Foley, et al., Addison-Wesley, 1995.) Of course, display of a single 2Dslice within a 3D frame is a subset of the general 3D reconstruction andvisualization system described above.

Alternative Beamforming Techniques

The embodiments described above use image data frames to collect datafor display and tracking frames to collect data to determine relativemotion of the transducers. Since the apertures for the image data framesand the tracking frames are physically separated, the collection ofthese frames of image data really form two independent beamformer tasks.If desired, multiple simultaneous transmit and/or receive beams can beused to reduce the time required to accumulate tracking data. Oneexample of a multiple transmit-beam system, which could readily beadapted for simultaneously transmitting ultrasonic beams for the imagedata frames and the tracking frames is disclosed in U.S. patentapplication Ser. No. 08/673,410, assigned to the assignee of the presentinvention. Similarly, a multiple receive-beam beamformer can be used toprovide a separate receive beam for each transmit beam, thereby speedingdata acquisition. The multiple receive-beam beamformer disclosed in U.S.patent application Ser. No. 08/432,615, assigned to the assignee of thepresent invention, can be adapted for this purpose. See also themultiple receive-beam beamformer disclosed in O'Donnell U.S. Pat. No.4,886,069.

The coherent image formation techniques disclosed in U.S. patentapplication Ser. Nos. 08/419,595 and 08/418,640, Wright et al., “Methodand Apparatus for Coherent Image Formation” (assigned to the assignee ofthe present invention) can be used advantageously with selectedembodiments of this invention. The system of the Wright patentapplications provides phase alignment between adjacent scan lines in thenear field, and, therefore, provides a shift-invariant speckle pattern.Such a shift-invariant speckle pattern should provide superior imagemotion detection based on the speckle tracking than that attainable witha conventional system in which the speckle pattern is shift variant dueto a lack of phase alignment between scan lines.

Very low cost scanning is possible using a frequency sweep technique toperform an angular scan. Consider the system shown in FIG. 41, in whicha tracking array is provided with a plurality of transducer elementsseparated by a pitch P. Each of the transducer elements is connected viatime delay elements to an input signal. As shown in FIG. 41, in thiscase the input signal is a relatively narrow band pulse train comprisinga tone burst 5 to 10 cycles in length having a fundamental period 1/F.Thus, transducer element n receives the input signal shown in FIG. 41delayed by delay interval ΔT·n. The angle of the main lobe of theresulting beam is determined by the duration of the time delay ΔT andthe period of the tone burst signal 1/F. By changing the frequency ofexcitation F, the angle of the scan line can be varied. Hence, for verylittle cost it is possible to scan the region. Of course, axialresolution of the resulting beam is not optimal, since for any givendirection the signals are narrow band and hence have a long duration. Inprinciple, it is possible to transmit with a broad band pulse, and thento use separate band pass filters for different frequencies so as toform multiple beams simultaneously. In the system of FIG. 41constructive interference, and therefore the primary beam direction, isachieved when P·sin θ is equal to v/F where P, θ and F are as shown inFIG. 41 and v is the speed of sound in tissue. By sweeping the frequencyof the input signal, and therefore the period 1/F, the resulting beamcan be caused to sweep in angle θ.

Another approach, as shown in FIG. 42, uses a set of mutually delayedsignals, such as four signals in the example of FIG. 42. Each signal isa relatively narrow band tone burst, and in the example of FIG. 42 thevarious signals are delayed from one another by multiples of 90°.Typically, the length of the tone burst is related to the number oftransducer elements and the times the signal is repeated across thearray. For example, a 32 element array may use an eight cycle toneburst. As before, by varying the frequency of the input signal, thesteering angle is varied. This approach is similar to that of FIG. 41,except that the four input signals are directly generated rather thantime delayed with respect to one another in a progressive fashion asshown in FIG. 41. In this case, constructive interference occurs at theangle θ satisfying the following equation:

4P·sin θ=(1/F)·v,

where v is the speed of sound in the medium as discussed above.

In alternative embodiments three signals spaced at a mutual delay of120° may be used instead of the four-signal arrangement described above.However, four delay signals are often preferred for reasons ofavailability of components for producing 90° phase shifts in sinusoidalsignals. This approach provides adequate lateral resolution at a lowcost.

According to the method described above, first an ultrasound transduceris provided comprising an imaging and a tracking array of any of thetypes described above. Then a transmit signal is provided to eachelement of the tracking array, this transmit signal having a pluralityof cycles and a characteristic frequency. Then the frequency of thetransmit signal is varied to control the scan direction. As pointed outabove, the transmit signals may be progressively delayed versions of asingle input signal or they may be directly generated.

Alternative Operator Warning Messages

If one of the tracking arrays detects motion but the other does not,this condition may be taken as an indication that the tracking arrayshowing no motion is not in tissue contact. Preferably, the systemcontroller includes programmed logic that responds to this condition byalerting the system operator with a warning message, such as an audibletone or a visual message on a display screen.

For example, as shown in FIG. 43, a transducer can include a singleimage data array 112 and three tracking arrays 114. Tracking data fromthe three tracking arrays 114 is processed by a receive beamformer togenerate three sets of tracking images that are applied to a motionestimator. This motion estimator generates three motion estimates, eachassociated with a respective one of the tracking arrays 114. These threemotion estimates are applied to a majority voting logic block, whichcompares the three estimates for a common point in time. It isanticipated that from time to time one of the tracking arrays may losecontact with the body of interest. Typically, one of the end arrays willlose contact first. If one of the tracking arrays produces a falsemotion estimate, because it is not in contact with tissue, the majorityvoting logic block determines the false motion estimate since it is in aminority. Alternatively, the quality of a motion estimate can beassessed by comparing the minimum sum of absolute differences to themean some of absolute differences. Once an unreliable motion estimatehas been isolated, a corrected motion estimate can be supplied which isextrapolated or interpolated from the motion estimates obtained with theremaining tracking arrays.

Single Block Motion Estimators

Although the overall motion between adjacent image frames can beestimated based on a collection of motion estimates in small regions asdescribed above, it is also possible to estimate motion using a singlelarge motion estimation block. When a collection of motion estimates isused, it is preferred to detect translation. A component of rotation maybe inferred if the translations are not all similar. When a single blockis used for motion detection, it is preferred to calculate both thecomponents of translation and rotation directly from this block, Ingeneral, the axis of rotation between a reference frame and a subsequentframe may lie anywhere within the image plane or outside the imageplane. Fortunately, modest errors in the location of the axis ofrotation are tolerable, since, for example, a small rotation about anaxis well outside the image frame causes a detected motion which ispractically equivalent to a pure translation, If a single block is usedfor motion detection, then the following algorithm can be used:

1. Select a subset of the image frame pixels from a first or referenceimage frame. This subset is selected to ensure that the subsequentcomputations are manageable. If the reference frame is 400×300 pixels,then a block 160×160 pixels may be selected, centered at the center ofthe reference image frame.

2. Computational processing time limitations may make it preferable touse a smaller block such as a 40×40 or an 80×80 reference block for themotion estimate. In this case, the 160×160 pixel block is desampledafter low-pass filtering to prevent aliasing.

3. A matrix of test axes of rotation is then defined, as shown, forexample, in attached FIG. 39. Preferably, the number of test axes is aslarge as possible, given the constraints of computation time. In somecases many more test axes than those shown in FIG. 39 can be used.

4. A correlation search is then performed. This search comprises takingthe 80×80 reference block from the first frame and comparing it to thecurrent frame and testing for the minimum sum of absolute differencesassuming:

(a) Rotation of the test block or the current frame about each of thedefined test axes;

(b) For each test axis, rotating through a predetermined set ofrotations, for example, between −4° and +4° in two degree steps. Whenrotation is performed, it is preferred to interpolate pixel values inthe rotated block (whether the reference or current block) so that thepixel values directly overlie one another.

(c) For each test axis of step (a) and for each rotation of step (b),the pixels in one of the two blocks being compared are translated inboth the X and Y axes by up to ±10 pixels.

(d) For each test axis, each rotation, and each translation, the minimumsum of absolute differences is calculated between the reference blockand the subsequent frame. The best correlation, as determined, forexample, by the minimum sum of absolute differences, is then used todetermine both the translation and the rotation of the subsequent framewith respect to the reference block.

In this way, a single reference block can be used to determine bothtranslation and rotation. Preferably, the search for the minimum sum ofabsolute differences uses the previous estimate of motion as a startingpoint for the search for the current estimate of motion. This approachreduces the total time required for the calculation. Experimentation canbe used to determine the most efficient search in terms of the likelylocations for the axis of rotation, the likely range of rotation betweensuccessive frames, and the likely range of translations betweensuccessive frames. As an example, it is most likely that the axis ofrotation will lie in the region defined in FIG. 40, and it is mostlikely that rotation between successive frames will be small.

Any of the foregoing embodiments can be adapted to utilize a selectivelyreceived harmonic component for tracking, imaging, or both.

As is known in the art, harmonic imaging provides advantages in certainapplications. In one form of harmonic imaging, ultrasonic energyconcentrated near a fundamental frequency (such as 2 MHz for example) istransmitted into a target such as a tissue of a medical subject.Ultrasonic echo information from the tissue is selectively received suchthat a harmonic component of the echo information is processed. Thisharmonic component is concentrated near a harmonic of the fundamentalfrequency (such as the second harmonic, or 4 MHz in the foregoingexample). Harmonic imaging has in the past been performed both with andwithout added non-linear contrast agent. When non-linear contrast agentis added, it often comprises micro-bubbles that are carried by blood,and that radiate echo information at the harmonic frequency. In tissueharmonic imaging, non-linear effects in the tissue distort a componentof the transmitted acoustic signal, giving rise to harmonic (e.g.,second harmonic) components not present in the transmitted signal.

It has been observed that tissue harmonic images as described aboveprovide a particularly high spatial resolution and often possessimproved contrast resolution characteristics. In particular, there isoften less clutter in the near field. Additionally, because the transmitbeam is generated using the fundamental frequency, the transmit beamprofile is less distorted by a specific level of tissue-related phaseaberration than would a transmit beam formed using signals transmitteddirectly at the second harmonic.

In order to enhance the harmonic image, it is preferred to shape thetransmitted signal in the frequency domain such that substantially noharmonic energy is transmitted. This can be done using a suitablyprogrammed transmit beamformer as described in U.S. patent applicationSer. No. 08/771,345, filed Dec. 16, 1996 now U.S. Pat. No. 5,696,737issued Dec. 9, 1997 and assigned to the assignee of the presentinvention, or a suitably filtered transmit beamformer as described inU.S. patent application Ser. No. 08/893,287, filed Jul. 15, 1997, andnow U.S. Pat. No. 5,833,614 issued Nov. 11, 1998, (Attorney Docket Nos.5050/218, 5050/219, 5050/220 and 5050/221).

Many techniques are available for selectively receiving the harmoniccomponent of the ultrasonic echo information. For example, a filter maybe included in the receive signal processing path to block thetransmitted fundamental component and to pass the desired harmoniccomponent. This filter may be a time varying filter of the typedescribed by Green, U.S. Pat. No. 4,016,750. Alternately a fixed, lowpass filter may be used on a demodulated signal, wherein thedemodulation carrier signal is selected to place the lower side band ofthe mixed signals such that what was the harmonic component in the RFdomain now lies within the pass band of the fixed pass band or low passfilter. The mixed frequency may be at base band (centered at 0 Hz) or atsome intermediate frequency (such as 2 MHz).

Harmonic imaging techniques can for example be used in the embodiment ofFIG. 22. In this case, the transmit beamformer 15 produces shapedtransmit signals for the transducer arrays 18, 20, 22. These shapedtransmit signals concentrate the transmitted ultrasonic energy near thefundamental frequency f₀, while substantially blocking transmittedultrasonic energy near the harmonic (e.g., 2f₀).

Received signals from the transducer arrays 18, 10, 22 are filtered asexplained above by the filter 13′ included in the image data signal pathand by the filter 13″ in the tracking data signal path. The filters 13′,13″ may be bandpass, low pass, or high pass filters.

In one embodiment the filter 13″ is selected to pass the second harmoniccomponent of the received signal and to block the fundamental component.Using the harmonic signal for the purpose of generating trackinginformation has the advantage that higher resolution and better contrastdata is provided, and hence more accurate and reliable estimates oftransducer motion are obtained. Tracking information may includemodulated RF data or demodulated (e.g. I/Q) data, either before or afterscan conversion. During tissue harmonic imaging, the target ismaintained free of additional non-linear contrast agent throughout anentire medical ultrasound examination session that includes acquisitionof the image data sets and the tracking data sets described above.

In another embodiment, the filter 13′ is selected to pass the harmoniccomponent and to block the fundamental component of the received signalsuch that the image data sets are created as a function of the harmoniccomponent. By employing harmonic signals for generating image data setsa higher quality image is obtained for display, as for example in a 3-Dvolume image data set.

Many variations are possible. For example, it is possible to use theharmonic component in the near field and the fundamental component inthe far field where penetration of the harmonic component is inadequate.This approach is discussed in detail in U.S. patent application Ser. No.08/904,825 filed Aug. 2, 1997 (Attorney Docket 5050/227), assigned tothe assignee of the present invention, and is applicable to both theimage data sets and the tracking data sets. As another example, acombination of fundamental and harmonic components (e.g. average values)for either or both the tracking and image data is used. Additionally,the harmonic component of the received signal may bemused for both theimage data sets and the tracking data sets. In some cases, it ispreferred to use harmonic components of different bandwidths for the twotypes of data sets; for example a narrower bandwidth may be used fortracking data and a wider bandwidth may be used for image data.

The harmonic component acquired by the image data array 18 may also beused for registering separate frames of image data to form an extendedview, using the techniques described above. The harmonic component maybe used either for the image tracking function or the image displayfunction, depending upon the application. As an example, when theextended image includes data collected at a far-field range, it may bepreferable to use the fundamental component of the received ultrasoundinformation for the image data sets, since the fundamental component ischaracterized by a greater penetration depth. Since it is not necessaryto perform image tracking at all depths (it may be adequate to calculateimage motion in the near field only), it may be preferred to use theharmonic component for the image tracking function since the harmoniccomponent has a higher spatial resolution and a higher contrastresolution and the comparison is being performed in the near field wherepenetration is not an issue. Other combinations of harmonic andfundamental components for the various combinations of tracking andimage formation can be made, including the use of a combination (e.g. asum) of fundamental and harmonic components for tracking or imageformation.

As mentioned above, there is an advantage in making multiple motionestimates and accumulating these such that those with the greatestquality or certainty are emphasized. Multiple motion estimates may bemade using fundamental and harmonic data and the results combined. Inthe very simplest case independently obtained motion estimates based ondifferent frequency components of the received signal are simplyaveraged to obtain a final motion estimate.

Conclusion

The systems and methods described above provide a number of importantadvantages. They are insensitive to the electromagnetic environment andto patient motion, in contrast to magnetic orientation methods. Thepresent invention is potentially implemented at lower cost than magneticmethods, and lower specification transducer arrays can be used.Potentially, the techniques described above can provide excellentaccuracy of registration, since accuracy is determined by imagingresolution which scales with frequency.

Depending upon the application, the improved transducers described abovemay require more or larger cables, and the footprint of the transducermay be slightly extended. However, these potential problems may beminimized by using the techniques described above.

Of course, it is intended that the foregoing detailed description beregarded as an illustration of presently preferred forms of theinvention, rather than as a limitation. It is the following claims,including all equivalents, which are intended to define the invention.

What is claimed is:
 1. A method for forming an extended field of view ofa target, said method comprising the following steps: (a) acquiring aplurality of sets of image information with an ultrasonic transducerarray, said array moved substantially in an image plane between sets ofimage information; (b) determining a component of motion in the imageplane based on a comparison of image information from a first one ofsaid sets with a second one of said sets and a search area; (c)adaptively changing a center of the search area within the second one ofsaid sets as a function of a previous motion estimate for step (b); (d)registering said first and second sets as a function of the component ofmotion; and (e) forming an extended field of view image as a function ofthe registration of step (d).
 2. The method of claim 1 furthercomprising step (f) of adaptively changing a size of the search area asa function of the previous motion estimate.
 3. A method for forming anextended field of view of a target, said method comprising the followingsteps: (a) acquiring a plurality of sets of image information with anultrasonic transducer array, said array moved substantially in an imageplane between sets of image information; (b) gating the acquisition ofstep (a) wherein first and second ones of said sets correspond to aportion of a cycle as a function of the gating; (c) determining acomponent of motion in the image plane based on a comparison of imageinformation from the first one of said sets with the second one of saidsets; (d) registering said first and second sets as a function of thecomponent of motion; and (e) forming an extended field of view image asa function of the registration of step (d).
 4. The method of claim 3wherein step (b) comprises gating to an ECG cycle.
 5. The method ofclaim 3 wherein step (b) comprises gating to a breathing cycle.
 6. Amethod for forming an extended field of view of a target, said methodcomprising the following steps: (a) acquiring a plurality of sets ofimage information with an ultrasonic transducer array, said array movedsubstantially in an image plane between sets of image information; (b)compressing the sets of image information; (c) transferring thecompressed sets of image information to a motion estimation computer;(d) determining a component of motion in the image plane based on acomparison of image information from a first one of said sets with asecond one of said sets with the motion estimation computer; (e)registering said first and second sets as a function of the component ofmotion; and (f) forming an extended field of view image as a function ofthe registration of step (e).
 7. The method of claim 6 furthercomprising step (g) of decompressing the sets of image information afterstep (c).
 8. The method of claim 6 wherein step (c) comprisestransferring to a remote site.
 9. A method for forming an extended fieldof view of a target, said method comprising the following steps: (a)acquiring a plurality of sets of image information with an ultrasonictransducer array, said array moved substantially in an image planebetween sets of image information; (b) comparing image information froma first one of said sets with a second one of said sets; (c) determininga component of motion with sub-pixel resolution in the image plane; (d)registering said first and second sets as a function of the component ofmotion; and (e) forming an extended field of view image as a function ofthe registration of step (d).
 10. The method of claim 9 furthercomprising: (f) determining at least a minimum and two neighboringvalues associated with motion; and wherein step (c) comprisesdetermining the component of motion with sub-pixel resolution in theimage plane as a function of the minimum and two neighboring values.